Lag with 100 parts

[klesh]

Active Texture Management (horrible!)

You're the second person I have read recently speaking negatively about ATM. What gives? I thought it was what everyone was using? Why is it horrible?

[Red Iron Crown]

You're the second person I have read recently speaking negatively about ATM. What gives? I thought it was what everyone was using? Why is it horrible?

It has far less effect now that stock textures and most mods have moved to the dds format, plus it often has side effects like blurry icons unless you are very careful in setting it up.

[Alshain]

You're the second person I have read recently speaking negatively about ATM. What gives? I thought it was what everyone was using? Why is it horrible?

Because the stock textures are now in the same format that ATM converts to, making it irrelevant. ATM is only useful if you have a lot of mods that have not converted to dds and are still using JPG or PNG. If that's the case I strongly suggest you reconsider that mod or just convert them to DDS yourself as it saves the annoying load time that ATM adds.

[cerberusti]

This may be true for the type of programs you write but it is not generally true and definitely isn't in the case of something like KSP. There are numerous complex calculations done by KSP on each frame which are definitely not memory bandwidth limited and could fairly easily be multithreaded with significant increase in performance on multi core cpus.

That last line is all kinds of wrong. There is nothing easy about writing a correct multithreaded program, KSP is very likely to be sensitive to memory latency (probably not bandwidth), and the performance increase from implementing multiple threads is likely to be minor (although we could disagree on the meaning of "significant".)

Which calculations do you think will see a large benefit from this?

If you are thinking the physics calculations I doubt it would benefit much (and this is the source of most performance issues in KSP.) It would probably work out ok on a rocket with multiple stacks which share only a single connection point, but those tend not to fly. Place some struts and the physics for that stack now relies upon the others, which means you are waiting on the result even with multiple threads. The overhead to make sure the result you need has been calculated and the related memory is not being updated can easily be more than you gain by multithreading (or you ignore that part and end up with a buggy program.)

There probably is some performance to gain by multithreading KSP, but there are probably also much better ways they could spend the time they are willing to dedicate to improving performance.

[cantab]

Back to the original question

KSP is unoptimized, that's what there's been a digression on.

Mods absolutely can hurt performance. I've seen as much as an 80 - yes, eight zero - percent fps drop from a buggy mod.

Your system is virtually identical to mine - I have a Phenom II X3 overclocked to 3.1 GHz, a GTX 750 Ti, 4 GB of RAM, and run Xubuntu 14.04. This means we can make a comparison. On a bone stock install, but with my normal game settings, I've been able to get the start of lag at around 100 parts. But that's just the real edge of it - the MET clock flickers yellow, with the game running at 0.9-1.0x normal speed and maintaining close to 25 fps. And it does depend on the parts, some affect performance worse than others. (For example, I flew the big stock plane and with the cargo bays open it was measurably worse than with them closed.)

If you're noticing that kind of slight framerate drop and finding it objectionable then KSP might not be the game for you.

If you're getting what I'd call serious issues, let's say below 15-20 fps, the MET clock solidly yellow or even red, something that a total non-gamer would notice, then I'd look at whether one of your mods is responsible.

[RandomRyan]

plays game with lag? me. (almost) all the games I play has lag.

1Gb RAM,Pentium E5300 clocked at 2.60ghz and yet, it has never stopped me from playing my favourite game.

who has intel integerated graphics? no one plays game with that!

anyway, how much should i amp the max physics delta thing?

i put 1 for now.

[haltux]

I am also a programmer who works on optimization of programs which process large data sets, so I have some insight as to what the possible bottlenecks actually are. I see a lot of complaints that games are not optimized well simply because they do not use all of the cores on the processor, but this is almost always due to a lack of understanding on the part of the user as to how a modern computer works and what the limitations are. The extra cores are there in case they are needed, they are not full processors. The only reason you have 4 cores is that it takes very little die space to do that compared to extra cache memory, but the vast majority of the time they are expected to be idle, even if your processor is maxed out.

The thing about the multiple cores your processor has is that they all share cache and memory, and very few programs are actually limited by the number of calculations your CPU can do. It is *far* more common for a task to be limited by either the amount of data which can be transferred from memory, or the latency of those requests (which can sometimes be hidden by clever programming tricks.) Multithreading a program which is limited by memory almost always gives you worse performance, and removes the ability to use certain very important optimizations (those threads compete for the same resources, and knowing what a thread is up to in relation to other threads requires very expensive synchronization commands.)

I am also programmer and I am daily writing code that are pushing all cores of all CPUs of all servers I am actually using. And I am not suffering memory latency/bandwidth bottlenecks. With all due respect I think you are over-generalizing your situation, and without deep knowledge of Unity (or similar game engine) internals you can't really confidently assert that using several cores will not lead to major performance boost. You may be right, and maybe not.

And everyone seems to agree on the fact that the bottleneck of KSP performance is physics, but as far as I know (please tell me if I am wrong) this has never been confirmed by the development team. And honestly I doubt it. I can imagine physics is a computation issue when you have tens of thousands of particles that you simulate independantly (see NVidia demos...). But for the kind of work that KSP requires, some hundreds of piece interactions at most, I don't see how that might overload our current PCs. And contrary to popular belief, many PC games are CPU bounded or at least partially CPU bounded, even when there is no physics involved. I have some ideas about what I am talking about, but I admit I am no expert, though.

So my point is that we don't even know where is the bottleneck. So arguing about the efficiency of multi-core optimization is a bit rash.

[magnemoe]

I am also programmer and I am daily writing code that are pushing all cores of all CPUs of all servers I am actually using. And I am not suffering memory latency/bandwidth bottlenecks. With all due respect I think you are over-generalizing your situation, and without deep knowledge of Unity (or similar game engine) internals you can't really confidently assert that using several cores will not lead to major performance boost. You may be right, and maybe not.

And everyone seems to agree on the fact that the bottleneck of KSP performance is physics, but as far as I know (please tell me if I am wrong) this has never been confirmed by the development team. And honestly I doubt it. I can imagine physics is a computation issue when you have tens of thousands of particles that you simulate independantly (see NVidia demos...). But for the kind of work that KSP requires, some hundreds of piece interactions at most, I don't see how that might overload our current PCs. And contrary to popular belief, many PC games are CPU bounded or at least partially CPU bounded, even when there is no physics involved. I have some ideas about what I am talking about, but I admit I am no expert, though.

So my point is that we don't even know where is the bottleneck. So arguing about the efficiency of multi-core optimization is a bit rash.

First of its an serious difference between thousands of independent objects who sometime collide, see Oblivion and Skyrim watermelon spawns and connected parts under load.

The watermelon spawn also created most lag then they was piled up in an huge heap, not then they was rolling down a hill.

The physic bottleneck should be pretty easy to point out, one issue is that the physic module is part of unity and run a lot under the hood even for the KSP developers, however lag scales with parts of ship, its also more lag in atmosphere as its extra physic calculations.

Complex models or large textures is not an serious lag issue, number of parts is, if parts are welded together lag is reduced.

Test: make an ship where the core is an large solid fuel booster, put lots of stuff on its surface, now replace it with an stack of 45 liter fuel tanks, now remove many of the side structure so part structure is equal, you will probably get far more lag as the tree is much deeper.

[Alshain]

I am also programmer and I am daily writing code that are pushing all cores of all CPUs of all servers I am actually using. And I am not suffering memory latency/bandwidth bottlenecks. With all due respect I think you are over-generalizing your situation, and without deep knowledge of Unity (or similar game engine) internals you can't really confidently assert that using several cores will not lead to major performance boost. You may be right, and maybe not.

And everyone seems to agree on the fact that the bottleneck of KSP performance is physics, but as far as I know (please tell me if I am wrong) this has never been confirmed by the development team. And honestly I doubt it. I can imagine physics is a computation issue when you have tens of thousands of particles that you simulate independantly (see NVidia demos...). But for the kind of work that KSP requires, some hundreds of piece interactions at most, I don't see how that might overload our current PCs. And contrary to popular belief, many PC games are CPU bounded or at least partially CPU bounded, even when there is no physics involved. I have some ideas about what I am talking about, but I admit I am no expert, though.

So my point is that we don't even know where is the bottleneck. So arguing about the efficiency of multi-core optimization is a bit rash.

We don't the developers to confirm it is physics, we can prove it through empirical evidence. It's the very reason weld mods work so well. The only difference between 10 parts and 10 welded parts using a welding mod is the lack of joints to simulate physics on. The welded parts are treated as a single item and it is an instant performance boost. Same textures, same graphical capabilities, but suddenly huge performance boost just by removing a bunch of joints. What else could it be?

[boxman]

Back to the original question

KSP is unoptimized, that's what there's been a digression on.

Mods absolutely can hurt performance. I've seen as much as an 80 - yes, eight zero - percent fps drop from a buggy mod.

Your system is virtually identical to mine - I have a Phenom II X3 overclocked to 3.1 GHz, a GTX 750 Ti, 4 GB of RAM, and run Xubuntu 14.04. This means we can make a comparison. On a bone stock install, but with my normal game settings, I've been able to get the start of lag at around 100 parts. But that's just the real edge of it - the MET clock flickers yellow, with the game running at 0.9-1.0x normal speed and maintaining close to 25 fps. And it does depend on the parts, some affect performance worse than others. (For example, I flew the big stock plane and with the cargo bays open it was measurably worse than with them closed.)

If you're noticing that kind of slight framerate drop and finding it objectionable then KSP might not be the game for you.

If you're getting what I'd call serious issues, let's say below 15-20 fps, the MET clock solidly yellow or even red, something that a total non-gamer would notice, then I'd look at whether one of your mods is responsible.

I have a phenom 2 945 x4 system as well and with older verisons it never ever was close to even FLASHING yellow at a part count that low. In fact you had to go 250-500 parts to even start seeing symptoms like the clock flickering to yellow and that was probably closer to 500 parts.

Now I see people with high end modern systems suffering lag with a part count where my old ancient phenom 2 had smooth frame rates.

This is something I started noticing even by watching KSPTV as with new versions they suddenly started having lag even with small ships.

If you had compared 1.x with lets say 0.22 you would see what a huge difference it is. Even my crappy ancient dual core laptop could handle 100+ parts before suffering lag.

Back then we were told they were optimizing the games as well as ram useage to make them peform better but in reality the exact opposite happened.

My fear is that unity 5 is now barely going to make up for the performance loss created by the bloat added by the new versions.

- - - Updated - - -

We don't the developers to confirm it is physics, we can prove it through empirical evidence. It's the very reason weld mods work so well. The only difference between 10 parts and 10 welded parts using a welding mod is the lack of joints to simulate physics on. The welded parts are treated as a single item and it is an instant performance boost. Same textures, same graphical capabilities, but suddenly huge performance boost just by removing a bunch of joints. What else could it be?

They have even confirmed time and time again that physics simulation is and has been a bottleneck for a long time.

[WeirdCulture]

Back to the original question

Mods absolutely can hurt performance. I've seen as much as an 80 - yes, eight zero - percent fps drop from a buggy mod.

do you have a name for us?

[cantab]

We don't the developers to confirm it is physics, we can prove it through empirical evidence. It's the very reason weld mods work so well. The only difference between 10 parts and 10 welded parts using a welding mod is the lack of joints to simulate physics on. The welded parts are treated as a single item and it is an instant performance boost. Same textures, same graphical capabilities, but suddenly huge performance boost just by removing a bunch of joints. What else could it be?

Some people have instead been pointing the finger at the resources code - the part that handles fuel consumption, electricity generation, and so on. That should be testable by comparing different ships with the same overall part count but varying numbers of tanks, engines, and structural parts.

I have a phenom 2 945 x4 system as well and with older verisons it never ever was close to even FLASHING yellow at a part count that low. In fact you had to go 250-500 parts to even start seeing symptoms like the clock flickering to yellow and that was probably closer to 500 parts.

Oh, absolutely. From 0.90 to 1.0 was a huge, huge, step backwards in performance. Unfortunately basically every modder moved their efforts to 1.0.x before this really became widely known. So we're faced with a choice between bad performance or being stuck with old mods.

do you have a name for us?

It only affected a couple of versions of the mod and got fixed. But I'll give you a hint: it's not called Kalileo Edited October 28, 2015 by cantab

[Alshain]

Some people have instead been pointing the finger at the resources code - the part that handles fuel consumption, electricity generation, and so on. That should be testable by comparing different ships with the same overall part count but varying numbers of tanks, engines, and structural parts.

Well it certainly can be more than one thing, but welding parts work. I don't need to prove that any further than it has been for the last several years. So we KNOW the joint physics are causing massive performance hit. Welding parts doesn't change the fuel logic.

[cerberusti]

I am also programmer and I am daily writing code that are pushing all cores of all CPUs of all servers I am actually using. And I am not suffering memory latency/bandwidth bottlenecks. With all due respect I think you are over-generalizing your situation, and without deep knowledge of Unity (or similar game engine) internals you can't really confidently assert that using several cores will not lead to major performance boost. You may be right, and maybe not.

And everyone seems to agree on the fact that the bottleneck of KSP performance is physics, but as far as I know (please tell me if I am wrong) this has never been confirmed by the development team. And honestly I doubt it. I can imagine physics is a computation issue when you have tens of thousands of particles that you simulate independantly (see NVidia demos...). But for the kind of work that KSP requires, some hundreds of piece interactions at most, I don't see how that might overload our current PCs. And contrary to popular belief, many PC games are CPU bounded or at least partially CPU bounded, even when there is no physics involved. I have some ideas about what I am talking about, but I admit I am no expert, though.

So my point is that we don't even know where is the bottleneck. So arguing about the efficiency of multi-core optimization is a bit rash.

This is intended to be informative rather than argumentative, so please take it that way.

While I may have given a program from my current employment as an example (it makes a particularly good one), I am generalizing from a couple of decades of experience writing high performance code for everything from mainframes to industrial machinery. While I may not know the specifics of the KSP code base, unity, or write games for a living... I can make an educated guess as to what causes the issue, and have some idea as to what to expect in terms of scaling to multiple threads.

Server workloads which involve multiple concurrent requests are usually what we call "embarrassingly parallel". Each client is making a request which does not rely upon the results of other pending requests, which means scaling to multiple cores, processors, or servers tends to be easy and work well (and is a big part of the reason server processors frequently have many cores.) In the case of scripting languages (for say a web server) adding more threads also tends not to have the same negative performance impact due to cache pressure, as programs in languages higher level than C++ cannot be easily optimized to force efficient cache and memory use anyway.

At the opposite end of the spectrum are tasks where each calculation relies upon the immediately preceding calculation, and branches are unpredictable. For these tasks not only do they not scale at all to multiple threads, they will struggle to even use the multiple execution units within a single core (which do not require threading to use, it happens automatically.)

The physics calculations are somewhere in between, but will be much closer to the latter than the former (that analysis does not require looking at source code, we can determine this based upon what the problem is.) Maybe they could put the solar and aero calculations on their own threads, but even something like that which at first glance looks like it could easily be parallel may not be. When on the same thread they can load the part and do all of the calculations at that time, which has the benefit that the part information all of those calculations must reference will already be in the L1 cache, or maybe even in a register. If these are not on the same thread that information may not be in the cache at all, which means loading it from memory instead (which additionally evicts something else which may have been useful in a future calculation from the cache to make room.)

A quick googling for a chart to show just how serious a cache miss or lock acquisition can be for performance gave the following (these are rough numbers as hardware differs, but give a general idea as to what costs are.) Mutex lock / unlock in the table is what you need to do in order to safely access data from more than one thread (and also the part many programmers totally screw up or leave out, leading to race conditions and other nasty threading bugs which cause intermittent and hard to trace problems.)

L1 cache reference 0.5 ns

Branch mispredict 5 ns

L2 cache reference 7 ns

Mutex lock/unlock 100 ns

Main memory reference 100 ns

Compress 1K bytes with Zippy 10,000 ns

Send 2K bytes over 1 Gbps network 20,000 ns

Read 1 MB sequentially from memory 250,000 ns

Round trip within same datacenter 500,000 ns

Disk seek 10,000,000 ns

Read 1 MB sequentially from network 10,000,000 ns

Read 1 MB sequentially from disk 30,000,000 ns

Send packet CA->Netherlands->CA 150,000,000 ns

At an average IPC of 8.5 for a single core and a clock speed of 4ghz, a single cache miss costs around 3,400 instructions.

A lock and unlock on memory which needs to be accessed from more than one thread costs around 6,800 instructions (double that to 13,600 for two threads which need to do it), which is part of the reason a task needs to be highly parallel before it is worth trying to multithread it. The mutex numbers actually seem a little high to me, so that table may give a worst case.

KSP may gain a little bit from multithreading, but it is not likely to gain much in the scenario people mostly see an issue with (launching a single large rocket.) I do not get the impression they have spent a ton of time optimizing KSP, so there is probably some low hanging fruit they could go after rather than trying a difficult and error prone solution which will only help under specific circumstances.

[magnemoe]

Well it certainly can be more than one thing, but welding parts work. I don't need to prove that any further than it has been for the last several years. So we KNOW the joint physics are causing massive performance hit. Welding parts doesn't change the fuel logic.

Add that the fuel and other resource calculations is trivial compared to physic. Its also handled by KSP not by unity so it should be far easier to handle.

On the other hand KSP is very badly optimized regarding memory use, it appears to load far more into memory than it need it also have some memory leaks.

[Wild Cobra]

A question for you experts.

I have windows 7 on my tower with the i7-4790. I was getting loads of memory hard faults. I did three things, and barely see them any more. Went from 8GB of CAS11 memory to 32 GB of CAS9, turned off my hyper-threading in the CPU, and set my page file system to zero.

I assume my hard faults cleared up with the page file system setting, which I would have never done without going to 32 GB.

I didn't notice any performance difference in KSP except for the hyper-threading change. Did the page file setting help, or should I it back to recommended? I have no problems running my computer for the course of the day, and I turn it off at night.

[Padishar]

Add that the fuel and other resource calculations is trivial compared to physic. Its also handled by KSP not by unity so it should be far easier to handle.

On the other hand KSP is very badly optimized regarding memory use, it appears to load far more into memory than it need it also have some memory leaks.

If the physics takes more cpu than the resource handling then please explain why the framerate drops so significantly when enabling the fuels cells on the craft in this bug...

Actual measurements of the time taken by various functions inside KSP shows the fuel cell resource calculations taking 77% of the CPU time of the main thread when the cells are active.

[Korizan]

OP please define your lag as it can come in 2 forms.

First is raw calculating power.

What color is your clock ? Green / Yellow / Red

Second is lag induced by a graphics card which "should not" change the color of the clock but could produce roughly the same results.

I am running a Phenom II 965 myself and my computer doesn't even notice 150 parts.

As far as video cards well QTY-2 ATI 6850's which have zero issues running 2048x1152

Memory 16 GIG / and Vista 64 (yes people still run it)

Edited October 28, 2015 by Korizan

[cerberusti]

A question for you experts.

I have windows 7 on my tower with the i7-4790. I was getting loads of memory hard faults. I did three things, and barely see them any more. Went from 8GB of CAS11 memory to 32 GB of CAS9, turned off my hyper-threading in the CPU, and set my page file system to zero.

I assume my hard faults cleared up with the page file system setting, which I would have never done without going to 32 GB.

I didn't notice any performance difference in KSP except for the hyper-threading change. Did the page file setting help, or should I it back to recommended? I have no problems running my computer for the course of the day, and I turn it off at night.

I would set it back. There are some windows api calls which fail if you have the page file off, and it disables a few windows features. Windows is decent at choosing what to keep in memory, even if it sometimes evicts program data to cache file data.

[Enorats]

As is KSP is only using like 1/8 of my PC. It's fine with any craft sub 200 parts, but slows to near unplayability if there are two such craft within 2.5 km of each other. Any surface base is unusable, the computer all but explodes when I approach the simplest of bases with a rover nearby.

[Alshain]

If the physics takes more cpu than the resource handling then please explain why the framerate drops so significantly when enabling the fuels cells on the craft in this bug...

Actual measurements of the time taken by various functions inside KSP shows the fuel cell resource calculations taking 77% of the CPU time of the main thread when the cells are active.

There is no correlation there. Just because joint physics are a massive performance hit doesn't mean there aren't any problems in other systems. I get part count lag on large stations with no fuel cells at all.

I'm having a hard time believing we are still discussing this, part count lag has been a problem at least since docking was added and fuel cells were added in the last major patch. Not only were they added in the last patch but they use an all new Module tied to the new ISRU, which is the likely culprit of the fuel cell problem since solar panels don't cause the same lag. Just because fuel cells cause lag, doesn't mean joint physics do not, and joint physics apply to ALL parts, not just 3.

[Wild Cobra]

As is KSP is only using like 1/8 of my PC. It's fine with any craft sub 200 parts, but slows to near unplayability if there are two such craft within 2.5 km of each other. Any surface base is unusable, the computer all but explodes when I approach the simplest of bases with a rover nearby.

In post 17, I listed the Passmark results of my systems CPU with hyper-threading on and off. I see an improvement by disabling it. You might want to D/L a benchmark tester, and try it on yours. My overall CPU performance is up by 6.89%, according to Passmark, by disabling hyper-threading. It might make a difference for you.

[sal_vager]

This kind of thread really should be in Support, moving!

[Wild Cobra]

There is no correlation there. Just because joint physics are a massive performance hit doesn't mean there aren't any problems in other systems. I get part count lag on large stations with no fuel cells at all.

I'm having a hard time believing we are still discussing this, part count lag has been a problem at least since docking was added and fuel cells were added in the last major patch. Not only were they added in the last patch but they use an all new Module tied to the new ISRU, which is the likely culprit of the fuel cell problem since solar panels don't cause the same lag. Just because fuel cells cause lag, doesn't mean joint physics do not, and joint physics apply to ALL parts, not just 3.

I see a huge difference in framerate with RCS on or off as well. But then, for these large vessels I'm making, I use 24 of them on a Rockomax Brand Adapter 02, then typically have 16 or more of these assemblies.

But RCS on or off also makes a difference.

[cerberusti]

If the physics takes more cpu than the resource handling then please explain why the framerate drops so significantly when enabling the fuels cells on the craft in this bug...

Actual measurements of the time taken by various functions inside KSP shows the fuel cell resource calculations taking 77% of the CPU time of the main thread when the cells are active.

It is unusual to place this many fuel cells on a craft, which reduces the importance of the issue. It is probably as simple as they never tested it with so many, and are using an algorithm which is not efficient as the number of parts grows.

The way to fix that (if they are interested in doing so) is to change the algorithm to something that scales better, not to split it off to another thread. A more efficient algorithm may be far faster, it is not unusual to see speedups of hundreds or thousands of times when fixing an inefficient algorithm (for that calculation at least.)

The best case for the physics calculations is still going to require a lot of computation.