Unity 5.2.... optimized UI.... coming NOWISH.

[EladDv]

As far as I understand, this isn't a problem of not enough man-hours. It's a problem of coding a lot of unnecesary stuff. Squad didn't need to code the whole electrical thing the way they did it. They don't need to run physics calculation when there is no change affecting a vessel. How many man hours would it take to cut all that code from the game and do it in a simpler, less resource intensive way?

i would say from 6 months to a year. not to talk about breaking every mod, save and ship in existence.

plus KSP dosnt even touch the physics engine so really the devs have nothing to do (and none of the blame) unless they would start writing a new physics engine for KSP from scratch(while that can be done i dont see them putting the effort after so much time debugging Unity physics and the fact that the game is post-beta)

[regex]

Let's face it: when Squad started this project they were rank amateurs at game design and programming. Well, maybe not programming, but the way some of the public functions were done seriously hurt reusability (asteroids, dude, asteroids) and the fact that the craft's part tree is traversed for every resource call is a bit indicative of amateurism. That's not exactly a bad thing (learning is fun, fresh ideas, all that) but it does result in statements like we've seen recently in the dev notes to the effect of "the old code is pretty bad". Squad's job, now that we're in release-land, is to refine this poorly-performing game into something excellent, especially if they're doing multiple console releases, because this game isn't entirely release-worthy.

This isn't about little niggling bugs that we see in all software, it's about general performance, which KSP really needs some work on.

[Darnok]

Why ? This (what you say) is exactly how Lamborghini was born.

Yes, some people have guts. Enough to stop complaining and prove they can do better.

That doesn't make argument "do better or shut up" valid. That is only basic rule of economy, if someone does something bad and you can do it better you will make more money than he does

[godefroi]

let me put it as simple as it gets.

PhysX cant divide linked rigidbodies(parts) between threads thus single vessels will not be multithreaded operations.

They will however be able to get multiple vessels on different threads in a way that optimally 1 vessel per thread(although it might increase as their number grows).

In addition Unity 5 also automatically offloads non-physics stuff to other threads available and the overall overhead of the new PhysX 3.3 is lower.

Everything you wrote is true, but none of that implies that Unity 4/KSP 1.0 isn't multithreaded. It is, and those multiple threads will happily use multiple cores. The only thing of significance that is changing is that physics for different vessels can be on different threads. This is only really relevant in scenarios like docking where you have multiple vessels in the "physics bubble". Gains outside of those scenarios will come simply from optimizations (inside PhysX, inside Unity, and inside KSP). Things other than physics are already on multiple threads; this isn't changing.

[EladDv]

Everything you wrote is true, but none of that implies that Unity 4/KSP 1.0 isn't multithreaded. It is, and those multiple threads will happily use multiple cores. The only thing of significance that is changing is that physics for different vessels can be on different threads. This is only really relevant in scenarios like docking where you have multiple vessels in the "physics bubble". Gains outside of those scenarios will come simply from optimizations (inside PhysX, inside Unity, and inside KSP). Things other than physics are already on multiple threads; this isn't changing.

i maybe wrong but from my test in unity(NOT KSP) if you dont internally and purposefully tell it to split stuff on multiple threads it wont do so. i dont know what KSP does but from dev notes i got the impression that they are not multithreaded right now. i never said it wasnt with 100% confidence or claimed you cant do it (but MT physics)

[Padishar]

i would say from 6 months to a year. not to talk about breaking every mod, save and ship in existence.

plus KSP dosnt even touch the physics engine so really the devs have nothing to do (and none of the blame) unless they would start writing a new physics engine for KSP from scratch(while that can be done i dont see them putting the effort after so much time debugging Unity physics and the fact that the game is post-beta)

Your estimate is so far out it's scary, I think you are misunderstanding the point he was making. The resource production and consumption mechanisms in KSP are seriously non-optimal and shouldn't really have been written in the way they were. It's ok to quickly knock out code that works but is slow when you are prototyping things but the code really should be reviewed and replaced before release.

Fixing the performance of and bugs affecting the production and consumption of resources is not a very large job. Significant improvements could be made in a day or two. I know this for a fact as I wrote a replacement for the function concerned that significantly improves the performance, eliminates most of the garbage production and fixes numerous bugs a couple of months ago. This took me approx. 7 hours of coding and testing time (mostly testing time). This code has been donated to Squad for them to do what they like with so they can get this improvement in a few minutes by simply dropping the code in. I am also working at designing further optimisations that will improve the performance even more than the ones already done though I've not spent a lot of time on it so far (basically, caching lists of resource container parts so the whole ship isn't scanned multiple times per frame to work out which parts to add or remove resources from). I estimate I've done about 3 hours so far and it will probably take around 30 to 40 hours in total before I'll be happy with it...

[godefroi]

i maybe wrong but from my test in unity(NOT KSP) if you dont internally and purposefully tell it to split stuff on multiple threads it wont do so. i dont know what KSP does but from dev notes i got the impression that they are not multithreaded right now. i never said it wasnt with 100% confidence or claimed you cant do it (but MT physics)

My KSP sitting here has 29 threads. 29 is more than one, therefore, multithreaded.

[EladDv]

My KSP sitting here has 29 threads. 29 is more than one, therefore, multithreaded.

i think you are confusing multithreaded work on a CPU/GPU to multiple threads of different systems. the fact that NO commercial CPU has 29 hardcoded threads (Cores or HyperThreads) which suggests that you are referring to the second one(which is a solution for "simultaneous" sharing of resources with different programs)-and that one is not affecting performance in any significant percentage as opposed to the first one.

Edited September 9, 2015 by EladDv

[Red Iron Crown]

Threads, not cores. There is no practical upper limit to the number of threads a modern OS can run concurrently on a timeshare basis.

E.g. Chrome famously has one thread per tab.

[Darnok]

Do you see the difference?

One is whining and complaining, serving no purpose than to potentially make the whiner feel better, the other is offering helpful suggestions, and while those suggestions may not work, it is clear that the the intent is to be helpful.

One is also much easier than the other.

If you are not offering specific steps to address the issue, you are not being helpful, just negative. (Note: 'Fix it' and 'Make an optimization pass' are not specific steps, they are cop-outs that mostly ignore the difficulties involved)

Ok, you are right that calling someone medicore is bit harsh, but if you are saying later that they made game where you can build craft with hundreds parts and each part resources are checked per frame then it is hard to call that kind of skills professional. I understand that this is first game made by Squad, so some things can be forgiven.

Also I was wondering why in 1.0+ KSP is running slow and how much it is temperature checks and new aero fault. I hope they excluded per frame checks if part is inside cargo bay or not.

IMO we have here few people that would love to think about better algorithms, but for example I have no idea how KSP is made. I doubt it game is going to be open source, but some data/manual how things work would be great and Squad could implement best ideas just like they are making best mods stock.

[godefroi]

i think you are confusing multithreaded work on a CPU/GPU to multiple threads of different systems. the fact that NO commercial CPU has 29 hardcoded threads (Cores or HyperThreads) which suggests that you are referring to the second one(which is a solution for "simultaneous" sharing of resources with different programs)-and that one is not affecting performance in any significant percentage as opposed to the first one.

I'm not confusing anything; the KSP process is running multiple threads, which, on my system here, are being scheduled onto two cores. On my machine at home, they'd be scheduled onto four cores. Software doesn't need to do anything special to have its threads scheduled onto multiple cores (or multiple hardware threads), it happens by default. Unless the programmer specifically requests that it NOT happen, it will happen. To software, hardware threads look just like any other core. That is to say, a dual-core hyperthreaded CPU looks to your program just like a four-core CPU. It just doesn't perform as well as a CPU that has four actual cores and isn't hyperthreaded.

I'm 100% clear on how this all works. I've been a professional developer of software (and by that I don't mean HTML) for a couple decades now.

- - - Updated - - -

Threads, not cores. There is no practical upper limit to the number of threads a modern OS can run concurrently on a timeshare basis.

E.g. Chrome famously has one thread per tab.

Sure there is. Each thread has a stack (chunk of memory), and those stacks add up. When you run out of memory, you cannot add more threads. Furthermore, when you have more threads than cores that are actually performing work (i.e. they're being scheduled by the kernel), you end up doing a lot of context switching, which gets expensive when you have lots of busy threads.

Also, Chrome famously has one PROCESS per tab. A process can have many threads.

Edited September 9, 2015 by godefroi

[Red Iron Crown]

Sure there is. Each thread has a stack (chunk of memory), and those stacks add up. When you run out of memory, you cannot add more threads. Furthermore, when you have more threads than cores that are actually performing work (i.e. they're being scheduled by the kernel), you end up doing a lot of context switching, which gets expensive when you have lots of busy threads.

Also, Chrome famously has one PROCESS per tab. A process can have many threads.

Thanks for the correction, I always get processes and threads mixed up. It's the process that gets the full virtual memory allocation, not the thread, so you're absolutely right. Context switching is what I was referring to when I said "timeshare basis".

[EladDv]

I'm not confusing anything; the KSP process is running multiple threads, which, on my system here, are being scheduled onto two cores. On my machine at home, they'd be scheduled onto four cores. Software doesn't need to do anything special to have its threads scheduled onto multiple cores (or multiple hardware threads), it happens by default. Unless the programmer specifically requests that it NOT happen, it will happen. To software, hardware threads look just like any other core. That is to say, a dual-core hyperthreaded CPU looks to your program just like a four-core CPU. It just doesn't perform as well as a CPU that has four actual cores and isn't hyperthreaded.

I'm 100% clear on how this all works. I've been a professional developer of software (and by that I don't mean HTML) for a couple decades now.

From what i know about Unity (and take it with a grain of salt) i dont think widespread multiple cores/hyperthreads computing is happening and while it does split the task to threads it dosnt contribute to speed as much as other cores would as all the threads still get allocated time on one of the CPU cores. as far as my tests go i can get 30-35% of CPU usage max from KSP which means 1 core is fully loaded while another is running at half power and is handling other stuff like unity's underlying framework-in other words- unity 4.x dosnt support (you can do it but it's not advisable to interact with it) USER code running on multiple cores/hyperthreading

Edited September 9, 2015 by EladDv

[Renegrade]

Software doesn't need to do anything special to have its threads scheduled onto multiple cores (or multiple hardware threads), it happens by default. Unless the programmer specifically requests that it NOT happen, it will happen.

NB: Many, many, many older threading systems weren't capable of that. Lots of 'em had cooperative models (entirely useless) and lots of 'em were limited to one CPU per actual process (GNU's Portable Threads, etc, also highly useless). Usually POSIX Threads (pthread.h) or NT-era Windows (9x/3.x etc weren't SMP-aware at all) threads are SMP/SMT/multicore-aware and capable, but you have to be careful if you ever go "on safari" into the embedded or legacy worlds..

Sure there is. Each thread has a stack (chunk of memory), and those stacks add up. When you run out of memory, you cannot add more threads. Furthermore, when you have more threads than cores that are actually performing work (i.e. they're being scheduled by the kernel), you end up doing a lot of context switching, which gets expensive when you have lots of busy threads.

Indeed! Plus, a number of operating systems put hard caps on the number of threads per process. I seem to recall BSDI 3.x/4.x having a limit of 1024, for instance. A project at work that was going on such a machine was going to involve something like fifteen hundred threads, wherein I kindly pointed out that the kernel had that hard limit set... heh. The buzzword-compliants weren't too pleased with that.

Thanks for the correction, I always get processes and threads mixed up. It's the process that gets the full virtual memory allocation, not the thread, so you're absolutely right. Context switching is what I was referring to when I said "timeshare basis".

If you follow the history of this stuff, it should be easy to remember - processes came first, threads later. My old Slackware'95 disc isn't going to have a pthreads library on it (probably)~

[Carraux]

Ugrading to Unity 5.0 will NOT improve game performance that much. A 125+ part rocket will still lag.

KSP is - as we assume - poorly optimized. We would gain considerably more performance, if Squad would optimize the source code. Recompute data only if the conditrion have changed. Which mean, that most times it is sufficient to compute ONCE and store the result. Compute as less as you can. One golden rule in game programming. And it seems that Squad didn't follow it well. But I have hopes for the future...

[marach]

Ugrading to Unity 5.0 will NOT improve game performance that much. A 125+ part rocket will still lag.

Actually that should be one of the areas that will improve. The change over from PhysX2 to PhysX3 is quite a big one what with almost all the x87 code being dumped in the bin where it belongs. In all honesty single ships is where we should see some of the performance gains. They shouldn't be huge but ships 10-20% larger than now before slow down is a possibility.

[magnemoe]

Threads, not cores. There is no practical upper limit to the number of threads a modern OS can run concurrently on a timeshare basis.

E.g. Chrome famously has one thread per tab.

Will not treads give load balancing on cores?

Treads has two benefits, one is that the rest of the program does not have to wait for tread to be completed, mainly useful if tread waits for data from slower source like disk or net, second is that it will balance over multiple cores.

[Red Iron Crown]

Will not treads give load balancing on cores?

Treads has two benefits, one is that the rest of the program does not have to wait for tread to be completed, mainly useful if tread waits for data from slower source like disk or net, second is that it will balance over multiple cores.

I guess it depends on how the threading is handled. KSP is multithreaded now, but all the other threads spend most of their time waiting for the physics thread to complete a frame. So even though it is multithreaded (and those other threads can run on other cores) it behaves very much like a single threaded application.

[adsii1970]

Translation: I like complaining about it more than doing something about it.

We have a idiom where I'm from... "Put up or Shut up". You think you can do better than Squad? You think their programmers and artists are "mediocre"? Go make your own game and prove it, otherwise stop talking. It is getting really old, the dead horse has been beaten, now you are beating the dust that used to be the dead horse. Squad knows you disapprove.

[Disclaimer: I am not an employee of SQUAD, just a fan of KSP]

For all the complaining and moaning of how bad the game is, to be completely honest, none of us had the idea to even tackle creating the basic game back before SQUAD did. Had we had such a vision, someone would have already done it. The programmers at SQUAD came up with the concept, the code, and we, the consumer, have loved it (mostly) to the extent where there is literally as many KSP sites out there (both official and unofficial) as there are SimCity sites, and considering that SQUAD does not even remotely have the footprint that EA/Maxis had, this is quite an accomplishment. Yes, there are some things that need to work better and as the consumers, we have the right to let them know what we think we would like to see, but the insults have gotten beyond acceptable and even unhealthy.

I think the guys at SQUAD and the respective unaffiliated MOD builders have done a fabulous job so far. As their experience improves I am sure the game will improve. I first began playing KSP back in 0.17 (as a demo) and bought it through Steam when it was on sale at 0.20. Yes, there have been many bugs, but with each release (except 1.0), the game has gotten a lot better. 1.0 was a lot of hype and probably should have been delayed for extended testing. Other than that, it has been a great game.

Ah, the old stock argument to shut down any criticism of creative works. It never has and never will be remotely convincing.

No, but much complaining never accomplishes much. My great-grandmother had a saying: "either get off the pot or ...., but just don't sit there..." She had moved to the Oklahoma Territory as a child in 1882; never went to school above the 8th grade, nevertheless, she was a very wise woman. For those of you complaining, ever thought about designing a mod or plug-in to address an area of the game you feel needs improvement. Offer to assist in debugging script or in being a tester for the new program. Simply just do not complain about the programming without offering any real and tangible suggestions besides "go get a computer science/programming book..."

[regex]

KSP is - as we assume - poorly optimized.

As we know.

[godefroi]

From what i know about Unity (and take it with a grain of salt) i dont think widespread multiple cores/hyperthreads computing is happening and while it does split the task to threads it dosnt contribute to speed as much as other cores would as all the threads still get allocated time on one of the CPU cores. as far as my tests go i can get 30-35% of CPU usage max from KSP which means 1 core is fully loaded while another is running at half power and is handling other stuff like unity's underlying framework-in other words- unity 4.x dosnt support (you can do it but it's not advisable to interact with it) USER code running on multiple cores/hyperthreading

Yeah, it does. Unity supports it, KSP does it. The reason why you have one fully busy core and other less busy cores is because one of the threads is significantly more busy than the rest. This is extremely common, even among multithreaded applications. Because a single thread can't be split across multiple cores, you end up with one busy core and the rest are underutilized. In this case, the single busy thread is likely the PhysX thread, but I don't know that for sure. Unity 5 won't change this for KSP in any meaningful way; when there's only one ship, there will still be only one PhysX thread. Unity does not decide what threads run on which cores, your operating system does. If you have multiple cores (or multiple hardware threads, it all looks the same to the application), your threads will be divided up among those cores.

Also, multiple core computing is definitely "happening", with even mobile devices now having four or more cores. Every .NET application you run is multithreaded whether the developer wrote it that way or not (because the .NET Framework itself starts up extra threads). Same goes for every Java application. Multithreaded applications have been the norm for a long time, and even back when the user generally only had a single core, it was worth it, because most threads spend a majority of their time waiting for things to happen (data read from a disk, input from a keyboard, etc), the CPU can get on with other things while a thread is waiting.

- - - Updated - - -

NB: Many, many, many older threading systems weren't capable of that. Lots of 'em had cooperative models (entirely useless) and lots of 'em were limited to one CPU per actual process (GNU's Portable Threads, etc, also highly useless). Usually POSIX Threads (pthread.h) or NT-era Windows (9x/3.x etc weren't SMP-aware at all) threads are SMP/SMT/multicore-aware and capable, but you have to be careful if you ever go "on safari" into the embedded or legacy worlds..

Thank goodness it's been a long time since we had to deal with that kind of nonsense.

[Carraux]

As we know.

Of course there is a lot of stuff we do not know. But I doubt, that the quality of the source code in those unknown spheres jumps to superior quality. I read the DevBlog regularily, among with other sources. And every time there is a sneack view into the internals of the game, my heart saddens (mainly because I am a software engineer for over 25 years or so, so I know a little bit).

[Renegrade]

Multithreaded applications have been the norm for a long time, and even back when the user generally only had a single core, it was worth it, because most threads spend a majority of their time waiting for things to happen (data read from a disk, input from a keyboard, etc), the CPU can get on with other things while a thread is waiting.

Actually, mostly because most programmers can't wrap their little minds around asynchronous design. select() has a timeout, y'all!

...and it's not like blocking on input is going to shut down the whole OS. Just a single process (and most processes don't actually have anything to do when waiting on input or data anyhow). Well, assuming you aren't under OS/2 and ignoring input events or something.

Thank goodness it's been a long time since we had to deal with that kind of nonsense.

I wouldn't sound so relieved, it's just getting replaced by a whole different set of nonsense.

[regex]

Of course there is a lot of stuff we do not know. But I doubt, that the quality of the source code in those unknown spheres jumps to superior quality.

Hence, we know.

[Endersmens]

Quote that is relevant to this thread...

We're in the process of updating to it and evaluating how well 1.1 works on it.

Yes, he's talking about unity 5.2.