Devnote Tuesday: The Mother of all Merges

[SQUAD]

<figure data-orig-width="480" data-orig-height="92" class="tmblr-full">[IMG]http://40.media.tumblr.com/0ff9377afa68dc3ad3fe6844aaf1bd38/tumblr_inline_nxzf4p6pMM1rr2wit_540.jpg[/IMG]</figure>Hello everyone!

With 1.0.5 released and stable we switched gears and have invested a good amount of time to make sure that everyone switched over to the Unity 5 side of development and merging all the code changes from the Unity 4 branch (1.0.x) into the Unity 5 branch (1.1.x). These branches have seen separate development for the past six months, and we’re serious when we say that has led to the mother of all merges.

Fortunately Git allows us to easily merge different branches, but this merge was something different that your standard run-of-the-mill merge. Felipe (HarvesteR) bit the bullet and attempted to merge the branches, an operation that resulted in 390 conflicts on a total of over eleven thousand changed files. A larger problem was that due to optimization concerns many files had been moved on the Unity 5 branch, and these files weren’t properly identified as having been modified but instead were set as added and deleted files. This wouldn’t normally be a problem but given that many of them had received code changes in either Git branch it became one: these changes were not compared and therefore would be lost in the progress.

This was a larger problem that the code change conflicts themselves, because hunting down the changes in both versions (and then determining which version of the code should prevail) manually is nearly impossible. Fortunately Git provided us with an alternative method: rebasing.

Instead of grabbing all the accumulated modifications from both sides and trying to crash them together straight away, rebasing works by treating one side as if it had happened first, then replaying the changes from the other side on top of that, as if it had happened afterwards. It’s a brilliant tool, which meant that by rebasing the Unity 5 branch on top of the Unity 4 branch, we could have the changes from our Unity 5 overhauls done after all of the Unity 4 work as if it had been completed before any work on Unity 5 had even started. Not quite time travel, but it’s the close.

The rebasing tool took us from 390 conflicts and a lot of added/deleted files to 398 ‘both modified’ conflicts, which was a good place for Felipe to start. This was a task that had to be completed by one developer because if the changes are ‘pushed’ then the files lose their conflicted tags which makes them much harder to find and resolve. At this point an overlooked line of code would mean that a bugfix or new feature would simply disappear.

The next job was making the game compile and work properly: compile problems galore, from syntax problems from problematic merges, to the much expected integration problems of trying to make two six-month long trains of changes crash together and keep on going in a single rail. Fixing these issues meant Felipe and other developers worked through the weekend and yesterday’s Mexican national holiday. The good news is that it worked as the two branches are now merged into, which (hopefully) contains all the changes done on both sides.

Straight after this mother of all merges we moved back to finishing the user interface overhaul. Many tweaks are still left to be done but with the Flight scene done and the Editors a good part of the way there progress is definitely felt and made. The tracking station also received some attention, and a very old bug was fixed in the process: small orbits on distant planets would be visible if you were zoomed in on any planet which wasn’t particularly hard on the eyes, but did affect performance. Orbits you could never see were wasting CPU time to render as tiny blips. That should be fixed now. If you zoom in to Kerbin, you should only see orbits belonging to the Kerbin system. If you zoom out, planets will fade in, but not their moons, until you zoom into them.

While on his coding spree over the weekend Felipe also fixed the rendering of the new map nodes and put quite a lot of work in on fixing depth sorting issues which were occurring because when transforming the node positions to screen space coordinates, they were losing all depth information.

The fix for that seemed straightforward: the nodes needed to have their Z coordinates back, but that presented a problem. In map view, even though the nodes live in a scaled down layer, the distances are still quite huge. This is a problem because the interface canvas depth space is quite finite, so we needed a way to transform these potentially boundless distances down to a finite space.

Usually this is done by defining a ‘far’ plane. A distance considered to be at infinity (but which very much isn’t), so you can get a depth scalar value by dividing against that. That approach always bothered me, because it requires a fair amount of guessing, and mapping depth linearly like that would mean wasting most of the canvas depth space with… well, empty space.

Felipe worked out a curve using a graph plotter which is based on the really nice asymptotes produced by taking reciprocals of things. Starting from a reciprocal (1/x) function which approaches zero as x approaches infinity, he worked out a neat little function that transforms the camera-relative distance of objects in the world to the finite canvas space, in such a way that they never quite reach 100% depth. They slowly taper out as they approach infinity, and thus the depth space is filled out as optimally as possible.

Meanwhile Mike (Mu) continued work on the 1.1 version of part tools. It now supports package dependency loading, DLL loading and a fair few other bits. The KSPedia tools were also folded into the package so you can create entire plugin packages and package them as a single asset bundle. The KSP-side of part tools, to load and unload assets dynamically, is almost complete. Dynamic loading and unloading of assets in the game will probably not be in 1.1 yet as it requires a fair bit of work to get this fully functional however it will be coming.

KSPedia is well under development, and Dave (TriggerAu) has been working hard to improve the efficiency of the development process in this area, setting up scripts that convert vector files (images) into component pieces that will allow the Unity Engine to display them ingame. A basic web display is also being made to allow the QA team to review the content independently of the game, making sure that no typos or inconsistencies slip through.
While on the subject of update 1.1. We’re currently at a fork in the road: do we continue with the features we set out to add and delay the release or do we perhaps cut a few features to meet our internal deadlines. There’s a few things dependent on this decision, and that not only makes it a very important decision but also one that is rather ‘opaque’ for people not involved in the process. There are many things to consider and we’ll keep you up-to-date in the future devnotes. Jim (Romfarer) quoted Felipe in this regard: “there is no amount of pre-planning that can avoid having to revise the layout entirely”.

It’s a hard landing for our new producer, Joe “Dr Turkey”, who will introduce himself below and will try to give you some insight into the current state of how we look at the near future:[INDENT]
Well, hi! So this is me, some of you already chatted with me over EJ’s stream last Friday.

Basically I have been working behind the curtain for the past month to hit the ground running. I’ve been crazy busy getting to know the (truly fantastic) team, the community, and the more ‘technical’ state of things as well as the business and overall planning overview as a whole. There is so much to do that a list would probably take the space of the whole of this week’s devnotes to write. I was there for the birth of 1.0.5 to take notes and analyze how the team performed as a whole.

In a VERY general summary, right now the plan is to consolidate current projects, consoles and the 1.1 update as smoothly as possible. Hopefully as soon as the transition period ends, I’ll have the time and energy to start laying the groundwork for some secret hush-hush thingies for the far- far away future. On the lighter side, I’m trying to get things all good and set up for the Squadcast, which will have me as host and some surprise co-host. I got some fun plans for my tenure as Squadcaster, but their implementation depends on the status of other projects, job timing, some extra planning and organizing beyond my current capacity as a human-poultry hybrid. No worries though, soon I’ll have power over spacetime and I’ll be able to extend all of Squad’s days from 24 to 42 hours per day, and then the true TAKE OVER will begin. I mean, *gobble. gobble!!*

Looking forward to more sleepless nights in the name of KSP!

[/INDENT]

That’s it for this week’s devnotes. Be sure to follow us on our official forums, Facebook and Twitter. If you have any questions you can post them there. Edited November 17, 2015 by KasperVld
layout issue

[Glaran K'erman]

Very dramatic/excellent devnotes and welcome to gobbles! Edited November 17, 2015 by Glaran K'erman

[NoMrBond]

[quote name='SQUAD']Dynamic loading and unloading of assets[/QUOTE]

fffffFFFFF BIG RED FIRETRUCK

I mean, yes, excellent.

[Whirligig Girl]

Neato. Lots of great news here today.

[Hobbes Novakoff]

So, dynamic loading eventually? My question is, what are you dynamically loading? The building interiors and such I see the point of, but planets and parts are going to always have to be loaded, as the game has no idea if a craft will appear out of nowhere the contains a part that isn't loaded yet, or if your craft will suddenly warp off to another planet.

[KerbMav]

I did not get even half of the first half - but ... ok ... :D

So, the new producer will spoil our christmas by delaying 1.1 yes? :wink:
My totally irrelevant opinion: Get 1.1 done, completely, especially 64bit and stuff, if not christmas (strange time to release anyway, since 1.1.1 and 1.1.2 wont be worked on during the holidays, yes? :wink: ) than early January 2016; but I think releasing without some [I]promised[/I] content ... nyaah ...

[Hotblack Desiato]

dynamic loading/unloading sounds great. no more active texture management with compressing all the textures (which works quite fine, but reduces the quality of textures).

together with 64 bit support, this is going to be fascinating.

and with 1.1 in the not so far future, let's see how it will be going :)

[DMagic]

[quote name='Hobbes Novakoff']So, dynamic loading eventually? My question is, what are you dynamically loading? The building interiors and such I see the point of, but planets and parts are going to always have to be loaded, as the game has no idea if a craft will appear out of nowhere the contains a part that isn't loaded yet, or if your craft will suddenly warp off to another planet.[/QUOTE]

I think it's referring to loading and unloading of new assets. Like I add a bunch of new parts and plugins to the GameData folder and load them into KSP without having to restart. You can already do this with parts and config files (to an extent), but reloading plugins means restarting KSP. Do this a few hundred times while working on a mod and it gets a little old.

It sounds like something more related to mod development (and regular development) rather than regular player use.

[basic.syntax]

The code merge sounded massively headache inducing, even the complications had complications. Props for seeing it through, with most of your sanity intact!

[Tourist]

[SIZE=2]I was with you up to "Hello everyone!", after that I was in over my head. But, yay for programming stuff... that's good, right?

I did pick up that 1.1 may be further delayed. Not that there is anything wrong with that, gotta get it right, that takes time. I assume the dial has moved back from "coming soon" till "when its ready", we can just be thankful it hasn't slipped back to "god knows".[/SIZE]

[KasperVld]

[quote name='Tourist'][SIZE=2]I was with you up to "Hello everyone!", after that I was in over my head. But, yay for programming stuff... that's good, right?

I did pick up that 1.1 may be further delayed. Not that there is anything wrong with that, gotta get it right, that takes time. I assume the dial has moved back from "coming soon" till "when its ready", we can just be thankful it hasn't slipped back to "god knows".[/SIZE][/QUOTE]
Well the development is speeding ahead, but when you think of it every part of the game has to be tested and retested extensively. Almost all of the user interface was redone in some way, and making sure everything still works as expected is going to take a very long time. We'll see though, fortunately I don't have to make the call :)

[r4pt0r]

What is the good doctors area of expertice? Is he a coder? or more business end like max was?

[mcirish3]

Hmm...Well I have mixed feelings. 1.1 I believe has to date, had the longest development time of any update, approaching nearly 1/5 of total game development. I would rather it came out sooner rather than later but I Think I would rather is came out finished rather than sooner even more. It is absolutely essential for the sake of the game that a stable 64 bit release happen at the next release. Pushing that back further will in all probability kill KSP for a lot of users. Not having a stable 64 bit has already cost KSP a lot of modders and users. In hind sight I hope squad can see that releasing a 1.0 non Unity 5, non full 64 bit, unfinished game has killed much of the game's momentum. By delaying the switch to unity 5 you have increased the time needed to switch over, to the detriment of the game and the players.

[YargJay9991]

Mmm... I wonder what the secret Hush Hush thingies are?

I think it might be weather or multiplayer (But that's no secret) or even other AI things.

What does the community think?

Happy Flying
YargJay9991

[sal_vager]

[quote name='r4pt0r']What is the good doctors area of expertice? Is he a coder? or more business end like max was?[/QUOTE]

Turkeys, obviously.

Christmas is coming after all....

[basic.syntax]

[quote name='sal_vager']Turkeys, obviously.
Christmas is coming after all....[/QUOTE]Think this is a hasty analogy... turkeys don't fare very well over the holidays, but... at least [B][I]one[/I][/B] does get spared by the President of the USA...

[pellinor]

[quote name='YargJay9991']Mmm... I wonder what the secret Hush Hush thingies are?
I think it might be weather or multiplayer (But that's no secret) or even other AI things.
What does the community think?[/QUOTE]

To be honest, I wouldn't bet too much on features. Rather something on the business side of things, like the console ports, 3d printing service, or a completely different game.

[Tourist]

[quote name='basic.syntax']Think this is a hasty analogy... turkeys don't fare very well over the holidays, but... at least [B][I]one[/I][/B] does get spared by the President of the USA...[/QUOTE]

Well, in Australia they usually fare better, many even have positions in Front Bench (and Shadow Front bench, I say for the sake of balance).

[blizzy78]

Hate to say it, but have you been rebasing [B]regularly[/B]? 390 conflicts sounds like a hell of a mess that shouldn't have happened in the first place. You can't just rebase at the end of a large chunk of work, you need to rebase regularly, say daily. Yes, you will be getting conflicts, but only a few per rebase, not a huge pile. Edited November 18, 2015 by blizzy78

[Superfluous J]

I was told there would be no math.

I'm gonna assume this was a nice meaty devnotes full of good information. It sure looks it!

[Dr_Turkey]

And now the trip to the White House makes more sense, doesn't it?

[quote name='basic.syntax']Think this is a hasty analogy... turkeys don't fare very well over the holidays, but... at least [B][I]one[/I][/B] does get spared by the President of the USA...[/QUOTE]


Well, I guess the latter. My area of expertise is [B][I]-REDACTED: FOR HER MAJESTY'S EYES ONLY[/I][/B]- of course, but then again, only on weekends.

[quote name='r4pt0r']What is the good doctors area of expertice? Is he a coder? or more business end like max was?[/QUOTE]

[COLOR="silver"][SIZE=1]- - - Updated - - -[/SIZE][/COLOR]

Well, I would quote [I]that [/I]girl, on [I]that [/I]commercial...

[quote name='pellinor']To be honest, I wouldn't bet too much on features. Rather something on the business side of things, like the console ports, 3d printing service, or a completely different game.[/QUOTE]

[SpacedCowboy]

[quote name='Tourist'][SIZE=2]I was with you up to "Hello everyone!", after that I was in over my head. But, yay for programming stuff... that's good, right?

I did pick up that 1.1 may be further delayed. Not that there is anything wrong with that, gotta get it right, that takes time. I assume the dial has moved back from "coming soon" till "when its ready", we can just be thankful it hasn't slipped back to "god knows".[/SIZE][/QUOTE]

Dude!,,, That ought to be your signature. Rep to ta.

[TheMoonRover]

[quote name='SQUAD']...the mother of all merges...[/QUOTE]

That sounds like quite a mess. Even messier than an office reconstruction.

[RazorWolf19]

I'd hate to say rush the update, but it seems like sooner would be better than later at this point.
I understand that there is a need to add new gameplay features on top of the fixes, but 1.1 is looking so massive that even releasing it as is would be sufficient. Nobody is going to complain about bugfixes optimizations; besides, it makes sense to release a "Holiday Update."

[federicoaa]

I would say to release 1.1 this year, unity 5 is more important than more features right now (IMHO)