Performance - bug?

[SSSPutnik]

This should perhaps go in some bug reporting area, but I am unsure if it is a bug or not.

I have a massive ship, yes it's pretty laggy.

However, if I start the game, switch to the ship, performance is quite good. I can do stuff pretty quickly.

But, if I send another ship to within render distance, performance drops badly, and it becomes very laggy.

If that second ship then departs render distance, the performance does NOT return back to what it was before, it stays laggy.

Any ideas?

[Kasuha]

It might help if you investigated it more thoroughly.

For isntance:

- does it improve if you go to space center and return to the ship?

- is performance still bad if you switch to the small ship? And then back?

One of possible reasons might be that as soon as you move the two ships together, your memory usage grows over the limit of physically available RAM and your PC starts swapping. It then may take a while for things to settle down after RAM demands decreased. So watch if you don't get some HD activity after you got the two ships together.

Or it may be the game's garbage collector having hard time freeing enough RAM for the game to run, especially if you use mods and are getting near the limit of RAM the game can use.

[Alchemist]

Well, when I was assembling my Jool-5 mission, one of the tankers approached the mothership too fast, so it flied by it at several hundred meters distance, got out of rendering distance, then approached it again, and the lag got so terrible that I had to restart the game. Also during that refueling operation, the framerate sometime was twice lower than other times with the same part count (identical tankers and the same mothership).

The fact that framerate during docking operations often was lower than during motership launch also seems strange.

I think it might have something to do with memory management: memory fragmentation might even be more of the issue than actual total allocated memory size.

But that's what I don't like about C# - there's no proper object destructor, you just leave it hanging until cleaner gets it, and the cleaner might do something stupid.

[SSSPutnik]

@Kasuha : I've not check that, but I doubt it, the machine has 16GB RAM and no other memory intensive apps running. Usually just Outlook and remote desktops.

Alchemist and you sound on the right track re memory management. Once it goes laggy it doesntnseem to unlag ever without game restart.

Edited May 9, 2014 by SSSPutnik

[Padishar]

I suspect there are a number of bugs in the core code that puts a vessel onto rails. There seem to be various issues in 0.23.5 that could be explained by this. I would check the output_log.txt for anything nasty around the time the vessel goes out of range (e.g. a null reference exception or similar) as this would interrupt the code putting it on rails so some important things might not get done.

Generally speaking, "mark and sweep" garbage collectors are very reliable but it is still possible to cause leak-like issues if you accidentally keep hold of a reference to an object that you shouldn't. All other objects it references, and so on, recursively, will not be collected. In event driven systems (where you generally pass an object reference to several other objects) this can happen even if you are quite careful...

[Alchemist]

One more thing about memory problems. It was because of ISA mapsat (the last version is a bit glitchy), but I once saw KSP crashing with couple hundreds MB RAM below the usual crash point. The log showed the typical "cannot allocate memory" problem. The only source of that I can imagine is fragmentation - there's bunch of memory available, but it can't find a chunk large enough to store something bigger.

I don't think there's a proper solution except for a proper 64 bit build. Other options are either extremely optimizing some memory structures (what is the case for some mods, but not for the general structure of the game) or cutting out on some memory-hungry assets (which ends in including even more assets anyway)

[Sirrobert]

@Kasuha : I've not check that, but I doubt it, the machine has 16GB RAM and no other memory intensive apps running. Usually just Outlook and remote desktops.

Alchemist and you sound on the right track re memory management. Once it goes laggy it doesntnseem to unlag ever without game restart.

Your RAM doesn't matter

KSP is a 32bit program, and it can only use 4GB or ram

[Padishar]

Your RAM does matter to whether or not KSP has to start using virtual memory. It should be possible to get up to 3.4 gig memory usage with only 2 gig of physical RAM in the machine but it will likely thrash the VM pagefile quite a bit. The important thing is whether the working set (the bits of memory the program has allocated that it is actually using, e.g. lots of part and planet textures that you aren't currently seeing can be swapped out of physical memory) fits into the available physical RAM. On a machine with 16 gig it is unlikely that any of KSPs memory would need to be paged out unless you are running other programs that are using lots of RAM and/or accessing large amounts of disk data making the disk caching use more memory.

[SSSPutnik]

What Padishar said.

I'll check swap today and maybe run performance profiling on the machine whilst I repeat.

The main craft is just over 1000 parts.

[Alchemist]

One more thing to consider:

The parts in the vessel cross-reference each other (parent link, children lists), but part modules cross reference with their parts. If when unloading the vessel the part references are just thrown out (and the actual part object don't exist for unloaded vessels, there's a saved part structure and part module config nodes instead), these crosslinks remain, don't they? (of course, memory hungry meshes are properly destroyed, these unity objects do have the destructor). Does the garbage collector understands these lost cross-referenced objects? If not, these will remain causing extreme memory fragmentation (because some other things in-between, like meshes, still are removed)

That's why I'd prefer to have proper destructor - when you know what objects are to be destroyed with this, you can do it with no problem. With the garbage collectror you never know what stray links may remain.

[Padishar]

That's why I'd prefer to have proper destructor - when you know what objects are to be destroyed with this, you can do it with no problem. With the garbage collectror you never know what stray links may remain.

It is swings and roundabouts really. Using a pure garbage collection based allocator should mean it is impossible for code to still have a reference to an object that has been deleted. This is one of the most common causes of access violations in programs written in C/C++ (after code simply not checking a pointer isn't null before dereferencing it).

Your example of cross references is not a problem for the Mono/.NET garbage collector. It is designed to correctly handle loops of references between objects, e.g. if you have a linked list of objects where each object has a reference to the previous and next objects in the list and you have a variable that contains a reference to the first object in the list then, when you set that variable to null (or it goes out of scope if it's local variable inside a function) then the garbage collector will realise that there are no external references to the data structure so all the individual objects in the linked list can be released. In a correctly written program this will not cause a significantly different amount of memory fragmentation.

It is no harder to write a correct C# program than it is a C++ program, in fact, it is usually easier as it requires less explicit boilerplate code to be written. The main difference is in how bugs manifest themselves. If you have a program that creates objects and passes references to them to several other objects, e.g. you register the object with a number of different callback providers, then, when it comes time to delete the object, you still have to remove the object reference from all of the callback providers or you will have problems. In C++, if you delete the object and have left it registered then you will be accessing memory that is no longer valid, it could have been reused for a completely different type of object or have been unmapped from physical memory and this would cause a crash. Unfortunately it may not have been reused yet and then the call can appear to work just using the potentially unsafe data in the object whose destructor has already been called. In C# this is impossible as if you leave a reference with a callback provider then the object will not get deleted so the memory can't possibly be reused or unmapped. Obviously, in each case, you need to do the same thing, unregister the object from all of the callback providers, then it will be safe to delete in C++ or it will be collected in C#.

One way of "automatically" handling deregistration in C# is to make your callback functions have a way of indicating that the object is no longer valid. Then, all you have to do is tell the object to do this when it is called and make the callback provider detect the special return value (or however the indication is done) and unregister the object. It means writing a little extra code in the callback function and the callback provider but also means you don't have to explicitly unregister the object from all the different providers it is registered with but you still can manually unregister the object if you don't want it sticking around in a "dead" state until all the callback providers have tried to call it.

TL;DR

Neither approach to memory allocation is "better", they both have advantages and disadvantages, but the level of memory fragmentation is not significantly different in either case. That is affected much more by changes to how the algorithm works than whether you are using garbage collected references or pointers...