CPU Performance Database

[DMagic]

I recently updated my video card from an AMD HD7750 to a GTX 750 ti and I figured I'd see if it made any difference here. As you can see the results are pretty much exactly the same. Everything else in the computer is unchanged, only the video card was upgraded.

Even though it doesn't affect such a heavily CPU limited situation like this though, the 750 ti is a huge upgrade over the 7750. I can run KSP with everything at max, no ocean terrain tweak nonsense, and pretty much stay at 60fps all of the time until I get to CPU limited situations. It's a really great card for people who have space constraints in their computer (there is a low-profile version available, which I have) or are limited by their power supply (lots of retail computers have crummy PSUs that can't handle more powerful GPUs) as it doesn't require any extra power cables.

[Xaiier]

Intel i5-4670k @ 3.4GHz

16GB DDR3-1600

GeForce GTX 760

Windows 7 64-bit

All settings at max except pixel light count and shadows, looking up enough to not see horizon, though I don't think GPU limitation is a problem.

https://dl.dropboxusercontent.com/u/99458555/KSP%202014-03-30%2020-18-03-88%20fps.csv

https://dl.dropboxusercontent.com/u/99458555/KSP%202014-03-30%2020-18-03-88%20frametimes.csv

If I have some free time I'll try it on my Macbook Pro and my HP Revolve 810. Does anyone know of a good FPS recorder for OSX? I can run KSP in windows, but I usually don't and the comparison would be interesting.

Edited March 31, 2014 by Xaiier

[Whackjob]

Gonna be running a AMD-FX-6100 processor when my memory finally arrives. Wonder how that'd stack up...

[DMagic]

Intel i5-4670k @ 3.4GHz

16GB DDR3-1600

GeForce GTX 760

Windows 7 64-bit

All settings at max except pixel light count and shadows, looking up enough to not see horizon, though I don't think GPU limitation is a problem.

If I have some free time I'll try it on my Macbook Pro and my HP Revolve 810. Does anyone know of a good FPS recorder for OSX? I can run KSP in windows, but I usually don't and the comparison would be interesting.

Thanks, I've added your results. For OSX I found this little utility for recording FPS, it's pretty simple and seems to work ok, it's what I used for my Mac results. I thought I had a link to that on the first page, but apparently I don't, I'll add it.

Gonna be running a AMD-FX-6100 processor when my memory finally arrives. Wonder how that'd stack up...

As far as AMD CPUs it should hold up fairly well, of course AMD processors don't do to well with KSP, but at the higher end they're usually good enough, for normal people that is.

Edited April 1, 2014 by DMagic

[Xaiier]

I must say I'm really impressed with how much overclocking my processor can help. Definitely will consider that more.

Do you think you could maybe re-color the charts? It's really hard to tell what's what with like 4 shades of blue.

I also ran a test on my laptop:

i7-3687U @ 2.10GHz Max 2.60GHz

12GB DDR3-1600

Intel HD 4000

Absolute min settings (1366x768).

https://dl.dropboxusercontent.com/u/99458555/KSP%202014-04-01%2014-52-06-57%20fps.csv

https://dl.dropboxusercontent.com/u/99458555/KSP%202014-04-01%2014-52-06-57%20frametimes.csv

I must say I'm fairly impressed.

Edited April 1, 2014 by Xaiier

[DMagic]

I must say I'm really impressed with how much overclocking my processor can help. Definitely will consider that more.

Do you think you could maybe re-color the charts? It's really hard to tell what's what with like 4 shades of blue.

Yeah, I think Excel uses some kind of color blind color scheme by default, so there is limited variety, I'll see what I can do. I'll also probably split up the results a bit more, it's hard to see more than five or six results on one graph, and while I'm at it I'll try to come up with some more interesting comparisons, not just grouping them together based on CPU family.

I tried out 0.23.5, I see very little difference in performance. However, the rocket does not appear to be stable. The second and third ring of SRBs took out engines during both test runs, I was able to complete both runs, but it was a little tricky and required separating some stages before they were out of fuel (that's a little scary). So if anyone wants to run this I would hold off until I can take another look at the rocket, I think maybe replacing the parachutes on the SRBs with sepratrons might be enough to fix it.

[Xaiier]

Yeah, I think Excel uses some kind of color blind color scheme by default, so there is limited variety, I'll see what I can do. I'll also probably split up the results a bit more, it's hard to see more than five or six results on one graph, and while I'm at it I'll try to come up with some more interesting comparisons, not just grouping them together based on CPU family.

While I agree, It does become hard to compare when there's multiple graphs. I wonder if there is an online resource you could use that could allow for custom selecting which ones you want to see and being able to trace the graph with your mouse.

EDIT: This looks cool! https://www.google.com/fusiontables/DataSource?docid=1ZqxbDN4wHoBO9Os4JnUx2RzpdL-5pyk8DY_jPF3B

EDIT2: A super quick attempt at shoving them all in there, just to see how it would look. https://www.google.com/fusiontables/DataSource?docid=1Vu56ybjEELg1eI1X1DkEuoMdokb9Iu8Q7D0ubJqr

I tried out 0.23.5, I see very little difference in performance. However, the rocket does not appear to be stable. The second and third ring of SRBs took out engines during both test runs, I was able to complete both runs, but it was a little tricky and required separating some stages before they were out of fuel (that's a little scary). So if anyone wants to run this I would hold off until I can take another look at the rocket, I think maybe replacing the parachutes on the SRBs with sepratrons might be enough to fix it.

http://i.imgur.com/TfZqHcq.jpg

It looks like a lot of the improvements were GPU side, so there will be major boosts for those who didn't have good graphics cards. Might be good to run your tests with the view changes again and compare those.

http://forum.kerbalspaceprogram.com/threads/74599

Edited April 2, 2014 by Xaiier

[DMagic]

While I agree, It does become hard to compare when there's multiple graphs. I wonder if there is an online resource you could use that could allow for custom selecting which ones you want to see and being able to trace the graph with your mouse.

EDIT: This looks cool! https://www.google.com/fusiontables/DataSource?docid=1ZqxbDN4wHoBO9Os4JnUx2RzpdL-5pyk8DY_jPF3B

EDIT2: A super quick attempt at shoving them all in there, just to see how it would look. https://www.google.com/fusiontables/DataSource?docid=1Vu56ybjEELg1eI1X1DkEuoMdokb9Iu8Q7D0ubJqr

Wow, that's a good idea, I'll look into that more soon (I'll also get your other results added sometime).

It looks like a lot of the improvements were GPU side, so there will be major boosts for those who didn't have good graphics cards. Might be good to run your tests with the view changes again and compare those.

http://forum.kerbalspaceprogram.com/threads/74599

Yeah, that seems to be the case. I'll need to switch back to 0.23 and run this again with different viewpoints, I might also need to cripple my new GPU since it seemed to be able to handle the ocean lag without many issues.

In the meantime I've been fixing my CPU rocket with some rather predictable results.

I think I've found a way to fix it though, replacing the parachutes and nose cones with sepratrons on the second and third SRB rings seems to make it stable, I just need to test it several more times. Hopefully the difference in part type won't affect performance.

[DMagic]

I updated the spaceport and mediafire links with the new version of the rocket. It turns out that the new joint system causes struts to hold on to radially decoupled parts a little bit tighter. This was causing some of the SRBs to crash back into the rocket. So I just struted the SRBs to themselves (the struts are nearly invisible now, but they are still there), this way the part-count and part types remain unchanged. Performance is the same and the rocket has been stable in the five or so times I've launched it.

A better comparison between 0.23 and 0.23.5 performance:

And I ran a quick camera-view test to see how much difference 0.23.5 made to the ocean terrain lag. I didn't get much of a decrease in performance in 0.23, so this difference will be more dramatic with weaker GPUs, but it is still quite visible. The blue and green tests are the standard, looking-up tests, while the orange and grey are the worst-case tests, with the horizon kept centered in the frame. The decrease in performance is still there in 0.23.5 (in orange), but it is almost unnoticeable for me. And in real-world cases, where I would have V-sync on and the framerate capped at 60, I would probably never see it. It is still a good idea to keep the camera pointed up while running the test though, this helps to remove a lot of GPU dependency, especially on weaker systems or laptops.

[HoustonDave]

I just upgraded my system from a 7-year-old HP AMD Athlon 2.x GHz Dual Core with 3GB RAM, to the following system:

Intel Core i3-4330 Dual Core Processor 3.5GHz

Gigabyte H87 LGA 1150 MOBO

8GB RAM

Samsung 840 EVO SSD

Mobo graphics only, no graphics card.

Installed Ubuntu and KSP stock 23.0 on it with no other software or mods. Would you like me to run a benchmark on it? It makes the HP look like a slug. ;-)

Edited April 11, 2014 by HoustonDave

[DMagic]

Installed Ubuntu and KSP stock 23.0 on it with no other software or mods. Would you like me to run a benchmark on it? It makes the HP look like a slug. ;-)

Definitely. The integrated GPU on that chip isn't half bad, so it should be ok. BuGLe is the only linux benchmarking tool that I know of, it can be a little tricky to use, but it seems to work well enough.

You should be able to use the new, 0.23.5 version of the rocket in 0.23, but it might give you problems (saying something like "this craft doesn't match your version of KSP"). If you don't already have it, here's a link for the 0.23 version: http://www./view/v8ctnadz540l7d9/CPU_Test_600_v23.craft

Edited April 11, 2014 by DMagic

[humptydump]

I suggest the following instructions for benchmarking on Ubuntu Linux (other Distros should be similar).

1. Download BuGLe from here: http://sourceforge.net/projects/bugle/files/bugle/0.0.20140104/bugle-0.0.20140104.tar.bz2/download

2. Install "scons" (apt-get install scons)

3. Unzip BuGLe.

4. Open a terminal in the BuGLe directory.

5. Run "scons" to compile.

6. Run "sudo scons install" to install.

7. Copy the example files from "doc/examples" in the BuGLe-directory to "$HOME/.bugle/".

8. Run KSP with the command: "BUGLE_CHAIN=logfps LD_PRELOAD=/usr/local/lib/libbugle.so KSP.x86_64"

9. You will find a logfile with fps rates in the directory, where you executed the command.

[humptydump]

Here a chart for my Phenom II X6 1090T (3200MHz regular, 3600MHz turbo) created by above method (ignore the units on the X axis, I could not yet figure out the time units of BuGLe).

[DMagic]

Thanks for those instructions on BuGLe, it's not quite as straightforward as FRAPS, so it's nice to have some help.

I think it outputs values as a framerate for each frame, which is kind of an odd way to do it. To figure out the x-axis you need to take 1/framerate for each frame, which should give frametimes in seconds per frame. Then, starting with the first frame add the frametime value to zero. Then add the next frametime value to the first product and repeat all the way down. That should give you the time in seconds where each frame takes place. Something like this in Excel, column A would be on the y-axis and column C would be the x-axis:

Also, can you send me the output log for your results humptydump? I can do all of the processing, but I want to add your results to the rest.

[Nemrav]

integrated ~$375 laptop(+tax) graphics here = 0.23 : below minimum graphics and minimal screen usage and sub 10fps,

0.23.5 = full screen, quarter res, terrain scatter and absolutely 0 water lag. 25 + fps

for a real good comparison you need to show both gaming computer and minimal computer I think....

sadly my last 0.23 copies are both modded so it might be rather pointless for me to graph the difference.

[humptydump]

@DMagic:

I have overwritten the logfile, but I am going to create a new clean one. Last night was just my plan to get BuGLe working. Of course that is even with my instructions not the easiest task, but at least a somehow experienced Linux user (who knows how to open a terminal and how to compile something) can now do it without reading tons of documentation. It took me literally hours to find out how to actually run it (preload the lib) and how to put a working filter file.

I am also planning to cross-compile a 32-bit version of BuGLe to run with KSP.x86. This would give us a nice comparison between the two on an exactly identical system.

[humptydump]

BTW: Would it not be better if somebody who knows the KSP API would just code a two-line mod which after each in-game second would output a real-time time stamp to the screen and/or a file?

This would be much more directly measure CPU/physics instead of FPS/graphics. I don't know the API, but this should be a 5-minute job for someone who knows it.

[humptydump]

I made a new run, here is my raw data:

http://spacepal.sagitta.uberspace.de/ksp/bugle.log

For less slowdown by the debug tool, I wrote the logfile into a tmpfs (RAM-Disk drive). The logfile starts with the launch of KSP, the launch of the rocket starts 90 seconds later.

Here is my own chart made from it:

http://spacepal.sagitta.uberspace.de/ksp/result-bench-phenomii-0235.png

Here is a little C program I wrote to convert the data from BuGLe into a list of framerates for each second:

http://spacepal.sagitta.uberspace.de/ksp/convert.c

Edited April 16, 2014 by humptydump

[DMagic]

Thanks humpty, I added your results and a few backlogged results to the first page. For now I just used Excel to convert the results in the log file to something usable, but I'll try out your script too.

It took me literally hours to find out how to actually run it (preload the lib) and how to put a working filter file.

I am also planning to cross-compile a 32-bit version of BuGLe to run with KSP.x86. This would give us a nice comparison between the two on an exactly identical system.

Yeah, I spent forever figuring this out then promptly forgot all about how to install it, so thanks for those instructions. And a 32-bit / 64-bit comparison would be great. Someone did that a while ago, but that was before version 0.23 and I'd be interested to see if something has changed.

BTW: Would it not be better if somebody who knows the KSP API would just code a two-line mod which after each in-game second would output a real-time time stamp to the screen and/or a file?

This would be much more directly measure CPU/physics instead of FPS/graphics. I don't know the API, but this should be a 5-minute job for someone who knows it.

Measuring the time scale difference might be a good idea. But it is inherently related to FPS, once you drop below 33 FPS the time scale is forced to expand, regardless of what is pushing you below that limit.

I also changed the charts a little and included some with more relevant comparisons. I like the online chart idea that xaiier suggested, but it's a little bit slow, I'll look into it more though.

[Xaiier]

BTW: Would it not be better if somebody who knows the KSP API would just code a two-line mod which after each in-game second would output a real-time time stamp to the screen and/or a file?

This would be much more directly measure CPU/physics instead of FPS/graphics. I don't know the API, but this should be a 5-minute job for someone who knows it.

Measuring the time scale difference might be a good idea. But it is inherently related to FPS, once you drop below 33 FPS the time scale is forced to expand, regardless of what is pushing you below that limit.

Since I'm kind of the KSP time wizard I could do such a thing, but I'm not sure how it would help. In the end what we want to see is a comparison between computers, and the FPS charts we have do that very well. It might provide some additional data, but it would still represent the same thing and have the same relative differences between computers.

It could help in data gathering though, as it would make the process a lot easier for every OS and standardize results. Let me know if you would like something like that DMagic.

I also changed the charts a little and included some with more relevant comparisons. I like the online chart idea that xaiier suggested, but it's a little bit slow, I'll look into it more though.

Yeah, that one in particular isn't the best as it is still in beta, but I'm sure there are others out there that are better. I just used it because it was simple and quick to use.

[humptydump]

Measuring the time scale difference might be a good idea. But it is inherently related to FPS, once you drop below 33 FPS the time scale is forced to expand, regardless of what is pushing you below that limit.

Yes, that is true, but there are several reasons for such a feature or mod:

1. It would make it easy to run comparable benchmarks on different operating systems and computers.

2. You do not have to setup external measuring tools on every system you want to bench, you can just run it everywhere where KSP runs in the first place.

3. Tools like BuGLe may slow down KSP a lot, because they measure the FPS for each frame, and write it down to a file for each frame. This can result in very different results on different systems, maybe even depending on the specific OpenGL code.

4. Also complicated tasks like building a 32-bit BuGLe on a 64-bit Linux for comparison measurements wouldn't be necessary any more.

Edited April 18, 2014 by humptydump

[DMagic]

I finally got around to more thoroughly testing the change in performance from 0.23.5. I ran several tests on my Surface Pro 2, which has a weak enough GPU to be significantly effected by the terrain detail.

I ran six tests, three on 0.23 and three on 0.23.5, all at the same graphics settings and all with the terrain quality set to default in the settings menu. Two tests were run in the standard, looking up mode (orange and yellow). Performance here is pretty similar.

The next set are more interesting; after applying the ocean terrain quality tweak I launched the rocket again, this time keeping the horizon centered throughout the entire run. You can see that the 0.23.5 run clusters together with the first, looking-up, tests (green). The 0.23 run shows a significant drop in performance though (grey), only improving when the vessel gets above 160km where the terrain switches to a lower detail mode (indicated by the blue arrows).

The last set are the most telling. With terrain quality set at default, and no changes made to ocean terrain settings, you can see that there is a huge drop in performance in 0.23 (brown). I was also watching CPU usage during the test (I run KSP in windowed mode) and I could see that after I was above 50 or 60km the CPU started throttling back. Normally it runs at about 50% utilization at 2.3GHz, during this test it dropped down to around 25% and the speed dropped down to around 1.5GHz. Clearly the game was GPU limited at this point.

When I tested it in 0.23.5 a completely different picture emerged (blue). Performance while keeping the horizon centered (which I've generally found to be a worst case scenario in terms of terrain related performance) was almost the same as it was while looking up, and looks almost identical the run with the ocean terrain settings tweak. Compare the blue and green lines, most of the difference appears to be from stage timing rather than performance. If you look carefully you can see that the framerate does begin to suffer as the part count drops. In both of the horizon runs under 0.23.5 performance is capped at around 30 FPS until the vessel gets above 160km, so I still recommend looking up while running the test.

Nevertheless, the performance change is pretty impressive. Above a certain level of GPU power you aren't likely to notice it, but with mobile computers and older graphics cards it can be a major, and very welcome, change.

TL;DR: Compare the brown line to the blue line; huge jump in terrain detail-limited performance.

Yes, that is true, but there are several reasons for such a feature or mod: ...

I'll see what I can come up with for this. It's definitely a good idea, but I don't want to make this the standard way of collecting results. There are several practical issues that come to mind when including a plugin that writes out results to a log file. But if I can come up with a way that gives results comparable to those given by standard benchmark software I will offer it as an optional method for collecting data.

Edited April 22, 2014 by DMagic

[Franklin]

Hope I did this right, here you go DMagic:

http://www.mirrorupload.net/file/0FJZKR8L/#!KSP+2014-04-22+19-22-44-12+frametimes.csv

CPU Model: AMD Phenom II X6 1100T

CPU Clock: 3.30GHz

[Xaiier]

I'll see what I can come up with for this. It's definitely a good idea, but I don't want to make this the standard way of collecting results. There are several practical issues that come to mind when including a plugin that writes out results to a log file. But if I can come up with a way that gives results comparable to those given by standard benchmark software I will offer it as an optional method for collecting data.

I'm curious as to what those might be.

[DMagic]

I'm curious as to what those might be.

The practical issues?

Is there some kind of user control, or does it run all of the time? Is it safe to allow it to run all of the time, will it affect performance, will it clash with something else? Should I rely on users to uninstall it after collecting data (a terrible idea)? Should it append results to a single file, create a new file for each run, or overwrite a single file? And so on.

None of these are really problems, I've already come up with something that seems to work and gives results that are identical to FRAPS. I think the best, and safest, solution is to make it as restricted as possible. The way I have it setup currently will only run when you are using my CPU rocket (it relies on the craft name, so any changes will cause it not to run) and only starts once you have activated the first stage.

I don't see any reason to collect any time data (game time vs. real time, fixedUpdateTime, ect...) because like you said before, it's all directly related to FPS anyway and that's the simplest method of displaying results. The framerate is the end result anyway and it's what actually affects how smooth or stuttery the game feels.

The plugin starts up in the flight scene, checks the craft name (and does nothing if it doesn't match), then starts a timer (based on the computer's clock, not Unity's) and collects the amount of frames rendered each second, outputting these to a single log file in the GameData folder (we aren't allowed to create files elsewhere). I need to test it to make sure it's working properly, gets destroyed without issues, and to see if works on multiple platforms.