Question about parallel branch development / updates

[JoeSchmuckatelli]

I am getting the impression that KSP2 is being presented to us as a series of drops from parallel development branches.  Possibly multiple parallel branches. This from reading posts by folks familiar with software development and observing trends with the look and function of certain aspects of the core game across what we've seen via ESA vids, Release and two patches. 

I'm happy with the thought - as it lets me believe that work can be moving along at pace on things like Science and Colonies, giving me hope for the game. 

At the same time they're discovering a lot of stuff we like / don't like (form and function of MNs, Burn Timer elements (DV, etc), Part Manager - and getting telemetry on all the stuff that is broken / not working as intended.  That stuff is getting fixed / fiddled with in the SB version. 

Question is - how easy is it to copy and paste the parts of the code that works - and that we like - into a parallel development branch?  When Science drops is it likely to be a fairly smooth move forward with the current fixes in place, or are we likely to see stuff we thought was fixed being broken again? 

 

 

[LoSBoL]

Well, I don't know how it is with other games development, it does look like multiple development branches being worked on in the same time if you look into steamdb

https://steamdb.info/app/954850/depots/

[Scarecrow71]

So, I can't answer this from a game development standpoint, but I do have development experience; I am a code monkey at work in my day job, responsible for developing SSIS packages for file ingestion and automation of file and data routing.  Lot of scripting, lot of SQL.  LOT of SQL.  Tons of stored procedures and database management.  The point is that I am a developer, and can answer this from my own personal standpoint.  And the answer, at least from my perspective, is that our packages and code are written in lower level environments (DEV, for example), and are designed specifically for allowing copy/paste of new code for about 75% of the enhancements that are made.  This would include/involve code, tables, stored procedures, packages, etc.  That 25% that falls outside of this are those enhancements that are simply touching a lot of different code scripts and packages.  You know, stuff that you really need to be careful with and probably hand-code most of it in the applicable environments to make sure you don't bone anything.

With that said, I can honestly say that there should be some percentage where the devs here are able to copy/paste from one dev branch to another...but what that amount is?  Unknown, at least by us.

[darthgently]

1 minute ago, Scarecrow71 said:

So, I can't answer this from a game development standpoint, but I do have development experience; I am a code monkey at work in my day job, responsible for developing SSIS packages for file ingestion and automation of file and data routing.  Lot of scripting, lot of SQL.  LOT of SQL.  Tons of stored procedures and database management.  The point is that I am a developer, and can answer this from my own personal standpoint.  And the answer, at least from my perspective, is that our packages and code are written in lower level environments (DEV, for example), and are designed specifically for allowing copy/paste of new code for about 75% of the enhancements that are made.  This would include/involve code, tables, stored procedures, packages, etc.  That 25% that falls outside of this are those enhancements that are simply touching a lot of different code scripts and packages.  You know, stuff that you really need to be careful with and probably hand-code most of it in the applicable environments to make sure you don't bone anything.

With that said, I can honestly say that there should be some percentage where the devs here are able to copy/paste from one dev branch to another...but what that amount is?  Unknown, at least by us.

Refactoring ad nauseam is a joy after previous devs had copypasta fever.

[Scarecrow71]

11 minutes ago, darthgently said:

Refactoring ad nauseam is a joy after previous devs had copypasta fever.

Agreed.  Can't tell you how many times I've had to correct someone else's code because they did the copypasta recipe and failed to make sure they copied/pasted not only the correct code, but all of it AND into the right spots.

[darthgently]

Best is to not copy paste at all

24 minutes ago, Scarecrow71 said:

Agreed.  Can't tell you how many times I've had to correct someone else's code because they did the copypasta recipe and failed to make sure they copied/pasted not only the correct code, but all of it AND into the right spots.

Best is to not copy paste in the first place, but properly factor in the first place so all places call the same stretch of code in a stored procedure or similar

[darthgently]

3 hours ago, JoeSchmuckatelli said:

Question is - how easy is it to copy and paste the parts of the code that works - and that we like - into a parallel development branch? 

Generally you never want to copy and paste.  As next time that functionally needs to be addressed you must  alter in several places.  Ideally all places call a common section of code in a library where you can do one stop maintenance

Edited April 24, 2023 by darthgently

[JoeSchmuckatelli]

7 minutes ago, darthgently said:

Generally you never want to copy and paste.  As next time that functionally needs to be addressed you must  alter in several places.  Ideally all places call a common section of code in a library where you can do one stop maintenance

I clearly don't understand how projects like this work; although I have worked on multi-author projects and do find it frustrating to try to incorporate everything into a single document that doesn't have internal conflicts or random assumptions that don't flow with everything else.

 

I'm just picturing the section of the code that deals with the Burn Timer (for example).  The BT was kinda functional during Release, worked really well during Patch 1 -- but lacked some functionality that people desired - and then in Patch 2 added that functionality (DV calc)... but then the various timers stopped agreeing with one another and the 'system' got wonky.  Similarly, the MN style and functionality of Patch 1 looks markedly similar to the ESA event videos; but notably different from what we got on Release.

To me it looks like someone grabbed a whole bunch of code related to one style or the other and dropped it into the patch, rather than something that was coded anew or revamped during the period between iterations.

 

Ultimately - I'm wondering how studios handle a situation like this - where presumably some teammembers are working on Science and Colonies content - at the same time that basic functionality is getting tweaked during SB updates.  At some point they have to have a merger of the two, right?

[K^2]

1 hour ago, LoSBoL said:

Well, I don't know how it is with other games development, it does look like multiple development branches being worked on in the same time if you look into steamdb

https://steamdb.info/app/954850/depots/

Steam depots don't have to be 1:1 with branches. In fact, I'd be confident in saying that most of the dev depots are probably coming from the same branch, but it would also be an absolute nightmare to manage if everything there came from the same branch.

It also depends a bit on what version control Intercept uses. For example, we see 0.1.0.0 and 0.1.1.0 depots. Under P4 or SVN, these almost definitely correspond to branches. Git, in contrast, has a good tagging systems, and both of these could be coming off something like release branch tagged something like 0.1.0.0 and 0.1.1.0 respectively. And, of course, if Git is used for code, there's probably still P4 or SVN for content, which would still be branched.

From there, things get interesting. The thing that makes git so fantastic for working with code is that you can switch between branches very easily in the same clone directory. If there is an experimental feature being worked on, I can throw it in a branch, and simply check out to it or back to main or dev whenever needed without having to re-download the entire repository. With P4 and SVN, it's more painful. I would actually have individual directories for each check-out on these. Consequently, creating a branch for an experiment is a complication. On P4, I'm much more likely to simply work with an experimental shelf than a branch. So the branching strategy is going to be very different.

So a typical P4 game project would have branches like: trunk, release, release_0_1_0_0, release_0_1_1_0.... and individual developers might have shelves off of any of these to work on experiments and fixes, wile  git project would typically have main, release (with tags 0.1.0.0, 0.1.1.0, etc) and then a whole bunch of branches that individual developers created for patches and experiments.

3 minutes ago, JoeSchmuckatelli said:

Ultimately - I'm wondering how studios handle a situation like this - where presumably some teammembers are working on Science and Colonies content - at the same time that basic functionality is getting tweaked during SB updates.  At some point they have to have a merger of the two, right?

For bigger features like that, you usually have them in the release branch and simply disable it for specific versions. There is no good way to drag development of something like this entirely in parallel with everything else due to conflicts. You'll spend more time fixing things that got broken because somebody else modified code that they can't even see, because it's in your experiment branch, than actually working on the feature.

Edited April 24, 2023 by K^2

[cocoscacao]

1 minute ago, JoeSchmuckatelli said:

At some point they have to have a merger of the two, right?

Yes. You have a tool called Git to track, merge, pick changes. And for all devs here:

[K^2]

14 minutes ago, cocoscacao said:

You have a tool called Git to track, merge, pick changes.

I wish it was always Git. In game dev you're practically guaranteed for at least some of your project to be on another version control, and if you think merge conflicts suck to fix in Git, P4 wants to have a chat with you. A few years ago, I was working on a pair of AAA games, one nearing release, the other in pre-production, and we were using the common engine code. And every time we'd take fixes from one game into another, it would take a literal month for engineers in charge of it to get through it.

Edit: Oh, and while I'm at it, the project I'm currently on, an even bigger AAA title, and we have our code and content split between Git, SVN, and P4. Thank the gods nobody suggested throwing Plastic into the mix.

Edited April 24, 2023 by K^2

[darthgently]

58 minutes ago, JoeSchmuckatelli said:

I clearly don't understand how projects like this work; although I have worked on multi-author projects and do find it frustrating to try to incorporate everything into a single document that doesn't have internal conflicts or random assumptions that don't flow with everything else.

 

I'm just picturing the section of the code that deals with the Burn Timer (for example).  The BT was kinda functional during Release, worked really well during Patch 1 -- but lacked some functionality that people desired - and then in Patch 2 added that functionality (DV calc)... but then the various timers stopped agreeing with one another and the 'system' got wonky.  Similarly, the MN style and functionality of Patch 1 looks markedly similar to the ESA event videos; but notably different from what we got on Release.

To me it looks like someone grabbed a whole bunch of code related to one style or the other and dropped it into the patch, rather than something that was coded anew or revamped during the period between iterations.

 

Ultimately - I'm wondering how studios handle a situation like this - where presumably some teammembers are working on Science and Colonies content - at the same time that basic functionality is getting tweaked during SB updates.  At some point they have to have a merger of the two, right?

Generally a small group or single person will be responsible for incorporating/gatekeeping patches into an internal test release that then gets tested internally by a dedicated test team.  As for multiple teams working on the same code base, they have to communicate.  A lot.  I don't necessarily mean big meetings, but at least a lot of mutual cubicle visits, agreements on standards, touching base on assumptions etc.  

Generally there will be a design doc early on that describes interfaces and other shared code.  It is a "living" document typically and every dev has the responsibility to not just keep up with it, but to point out inconsistencies and gaps in it.  Every dev has to be able to justify deviating from the doc and a major fair faux pas is to deviate from it without letting anyone know on code that is shared across devs and teams.  Of course, ideally, every dev also has daily active input on what constitutes the design doc on a healthy team

Having code that basically does the same thing scattered across the codebase is frowned upon as it means duplication of effort and increased maintenance complexity.  This is what libraries, functions, classes etc are for.  You code the functionality in one place that all the other code makes calls to.  This will almost always make a smaller footprint memory also for the final product.  Copy paste always smells of desperation

[cocoscacao]

@K^2  I'm not in games, but web (backend mostly), and Git is absolutely dominant. I was only using SVN to pull some source code from very old repos, but I have no working experience with other version control tools. Just out of curiosity... are there any benefits to them?

[JoeSchmuckatelli]

33 minutes ago, darthgently said:

Copy paste always smells of desperation

I'm starting to get that.  I used the phrase because I'm painfully unaware of how the industry operates - and thus, enjoying the discussion.

Trying, then, to understand some of what i've seen with KSP2.  I could go photo hunting - but the clearest indication is the ESA Maneuver Node, the Release Maneuver Node and the Patch 1 Maneuver Node.  Release was a blobby mess that looked nothing like ESA - and once it dropped, Patch one did look like ESA event MNs.  Patch 1 MNs were graphically different and felt different, and with Patch 2 they're acting different from Patch 1.  Additionally, the Burn timer - it also changed from not great with Release, working quite well in Patch 1 and then being graphically different and having disagreements with itself in Patch 2.

So how does something like that happen? 

My pure guess is that there's some kind of Build A / Build B thing going on and each patch is coming from a different, albeit parallel build - rather than having a single build that is getting branched off of each time.

 

 

Edited April 24, 2023 by JoeSchmuckatelli

[K^2]

10 minutes ago, cocoscacao said:

Just out of curiosity... are there any benefits to them?

Git doesn't handle art assets particularly gracefully. At least, not out of the box. And even when configured to handle these kinds of files, for artists, there is zero benefit. For code, though, Git is absolutely the best way to go, so more and more studios end up running code repo in Git and art in P4.

The reason we have SVN for some of the code is because the code base is, indeed, old. I don't know how much survived the re-writes and refactors, but the foundation is from the 90s.

[darthgently]

15 minutes ago, JoeSchmuckatelli said:

I'm starting to get that.  I used the phrase because I'm painfully unaware of how the industry operates - and thus, enjoying the discussion.

Trying, then, to understand some of what i've seen with KSP2.  I could go photo hunting - but the clearest indication is the ESA Maneuver Node, the Release Maneuver Node and the Patch 1 Maneuver Node.  Release was a blobby mess that looked nothing like ESA - and once it dropped, Patch one did look like ESA event MNs.  Patch 1 MNs were graphically different and felt different, and with Patch 2 they're acting different from Patch 1.  Additionally, the Burn timer - it also changed from not great with Release, working quite well in Patch 1 and then being graphically different and having disagreements with itself in Patch 2.

So how does something like that happen? 

My pure guess is that there's some kind of Build A / Build B thing going on and each patch is coming from a different, albeit parallel build - rather than having a single build that is getting branched off of each time.

 

 

Not sure.  It could be as simple as a version control fumble where older versions of some files were accidentally reinstated.  But in that case I'd expect a fairly simple and quick "oops-fixedit" spanning a few hours

[Scarecrow71]

9 minutes ago, darthgently said:

Not sure.  It could be as simple as a version control fumble where older versions of some files were accidentally reinstated.  But in that case I'd expect a fairly simple and quick "oops-fixedit" spanning a few hours

And see, we use DevOps (formerly Team Foundation Server) for the purpose of version control so we can get back to specific builds of code in the event something goes wrong.

[darthgently]

2 minutes ago, Scarecrow71 said:

And see, we use DevOps (formerly Team Foundation Server) for the purpose of version control so we can get back to specific builds of code in the event something goes wrong.

Who knows what happens?  But however it occurs it seems clear there is no real regression testing beyond the KSP2 $50 Volunteer Brigade

[Rosten]

2 hours ago, K^2 said:

Thank the gods nobody suggested throwing Plastic into the mix.

Had to use that before and it's absolute cancer.  It felt like it's based on subversion from how it (mis)behaves but I never looked into the details.  Not sure why the company was trying to use it, I assume they liked the UI for the less technical people or maybe it was just going around their circles at the time.  It was so slow that I gave up attempting to do the equivalent of bisects and eventually just ignored repository history entirely, and the server would get into states that required the vendor to fix, at one point being down almost a week.  Art was in perforce but as I draw stick figures my hell was plastic.

[Guest]

Unity projects often use Plastic for version control as it’s owned by the same company and really well integrated with the engine. P4 is much less common as it’s more expensive and the integration with Unity isn’t as good.

Plastic works well with lots of branches, scales to really big repos, and handles assets nicely. If the project is well managed and the devs do a bit of regular coordination to make sure they don’t trip on each other’s feet and sync up regularly it’s totally feasible to work on several branches in parallel. (If you don’t you can get yourself into a hilarious mess though )

(Personally I prefer Git for code but Git/LFS is kind of clunky with asset pipelines so I think most studios will eventually end up with either Plastic or P4, and AAA games will almost certainly be P4.)

[jost]

3 hours ago, K^2 said:

Edit: Oh, and while I'm at it, the project I'm currently on, an even bigger AAA title, and we have our code and content split between Git, SVN, and P4. Thank the gods nobody suggested throwing Plastic into the mix.

Ouch, this hurts just from reading. I hope your salary compensates at least  a little bit for this pain.

[K^2]

5 hours ago, Periple said:

P4 is much less common as it’s more expensive and the integration with Unity isn’t as good.

Glad I don't have to worry about that working custom engines. Any integration's custom anyways, so good or bad, at least we're not stuck with some 3rd party crap only because that's the only thing that's integrated.

4 hours ago, jost said:

I hope your salary compensates at least  a little bit for this pain.

[LoSBoL]

 

I don't know anything about coding or developing, but you guys are giving me as layman a grasp of how things come toghether! Thanks, I've learned again today.

16 hours ago, K^2 said:

t also depends a bit on what version control Intercept uses. For example, we see 0.1.0.0 and 0.1.1.0 depots. Under P4 or SVN, these almost definitely correspond to branches. Git, in contrast, has a good tagging systems, and both of these could be coming off something like release branch tagged something like 0.1.0.0 and 0.1.1.0 respectively. And, of course, if Git is used for code, there's probably still P4 or SVN for content, which would still be branched.

I thought the 0.1.0.0 and 0.1.1.0 versions  branches are for 'us' to download earlier versions of the Game through the Steam's  'Beta participation' options?  This goes for KSP as well, where you can revert back all the way to v1.0.5.

 

 

 

[Scarecrow71]

19 hours ago, darthgently said:

Copy paste always smells of desperation

In most cases, yes.  Where we (I) use it at work is generally when only a singular line or two of code is being updated, and at that only when it's only in one location in a script.  In that case, it's more ease of use than anything else (although we could technically promote the entire batch with DevOps).

[darthgently]

3 minutes ago, Scarecrow71 said:

In most cases, yes.  Where we (I) use it at work is generally when only a singular line or two of code is being updated, and at that only when it's only in one location in a script.  In that case, it's more ease of use than anything else (although we could technically promote the entire batch with DevOps).

I have to disagree.  "Ease of use" in the short term.   Pain in the butt maintaining later.  The only case I can ever see for non-factored code is for extreme, and rare, performance reasons where call overhead to a central function would just cost too much for the particular situation.  But in this case there should be a big comment spelling this out and every instance of the duplicated code should be tagged by a common unique comment allowing searching for its many locations easier when maintaining later