Page 1 of 29 12311 ... LastLast
Results 1 to 10 of 289

Thread: CPU Performance Database

  1. #1
    Capsule Communicator
    Join Date
    Apr 2013
    Location
    Brooklyn, NY
    Posts
    1,760
    Blog Entries
    1

    CPU Performance Database

    Data in the second post | Pre-0.23 data in my blog entry

    Updates for KSP version 0.23.5 - Works in 0.24.x
    New CPU Rocket available on CurseForge; only run the test with the latest version of the rocket.

    At some point everyone playing KSP learns that performance drops off significantly with increasing part counts. Whether this is learned through hitting a wall in-game or reading about it online, everyone realizes this eventually. What I want to figure out is how exactly various CPU’s handle crafts with high part counts. I’ve seen lots of vague comments about how such and such CPU performs fine with some arbitrary part count, but what I want is actual data.

    So I went ahead and made a CPU performance test rocket. Using all stock parts I created a stable, functional, 600-part rocket. It is setup so that performance can be measured at a variety of part count levels throughout the ascent.

    Not having a stack of CPU's available to test, I need others to download and launch the rocket so that I can collect a large amount of data on CPU performance.

    This is what I want people to do:

    Download the rocket from CurseForge.

    Change two values in the settings menu:
    Set the physics delta slider all the way to the right; a physics delta-time per frame of 0.03s.
    Turn off V-sync.

    Launch the rocket without any additions or modifications in a fresh session of KSP (don't launch or fly anything else before this rocket).

    Change the camera view to look up; move it enough to keep the horizon out of frame (this avoids most GPU limitations, check the fourth post for more on this), don't go into map view and don't open any menus (ESC, F3).

    Fly straight up until out of fuel.

    And this is what I want to collect:

    .csv file with FRAPS data
    CPU model
    CPU clockspeed

    New Rocket:

    The new version of my CPU rocket uses tweakable settings to reduce the fuel load during all stages. This greatly shortens the time required to run the test, from about 9 minutes to just under 4 minutes of in-game time. The staging has also been simplified, now the rockets for the next stage fire at the same time as the decouplers and sepratrons from the previous stage, this eliminates all of the double staging events required. Everything seems stable, I haven't had any issues, but let me know if anyone has problems.

    The fifth post has a more detailed look at the new rocket: http://forum.kerbalspaceprogram.com/...l=1#post554432

    A few small, but crucial, tweaks have been made for the 0.23.5 version of the rocket. The new joint system was causing struts to hold onto radially decoupled parts a little bit tighter, resulting in hilarious, but less-than-useful effects. Please download the new version for any testing on KSP V0.23.5. The rocket works ok in KSP V0.24.

    ------------
    For data gathering, Windows users can download and run FRAPS, the simple, free, frame rate monitoring software. FRAPS has a benchmark function that can be run to capture the frame rate during the entire launch (you can also capture frametime data); this data is then exported into a .csv file which can be opened with Excel or any other spreadsheet software.

    By default the benchmark is started and stopped by pushing F11. A green frame rate counter should flash in one of the corners for a few seconds when the benchmark is started, and a red counter will appear when you stop it. Actually displaying the frame rate on-screen can affect performance, so the counter disappears during benchmarking. By default, results are stored in C:\Fraps\Benchmarks.

    For OSX I've found a utility for recording framerate, it's fairly simple to use and seems to work ok. You can get it here: http://forums.anandtech.com/showthread.php?t=2255863.

    Linux users can try BuGLe if you can figure it out. It's not too difficult to use, but it does take some effort. This post has some instructions on how to set it up.

    As an alternative you can try my CPU logging plugin. It creates a log file in the 'GameData/CPUDatabase' folder with a recording of your framerate every second. You can just submit that text file to me if you want. Do note that the vessel name for the rocket must remain unchanged; it is "CPU Test 600 v23_5" without the quotes. Get the plugin here: http://www.mediafire.com/download/cc...seLog_v1.1.zip. The source code is on GitHub.


    ------------

    If you don’t know your clockspeed then you probably aren’t overclocking and the CPU model should be enough to figure it out. But, if you want to know the real value you can use CPU-Z, it’s a good tool for this. Don’t trust the speed given by task manager, this can be wrong.

    Other computer details probably aren’t necessary unless you think they are limiting; having 1GB or less of RAM for instance, or something like an Intel HD3000 GPU.

    By gathering this data, along with CPU vendor id and clock speed, I hope to create a usable database of CPU performance. This will allow people to estimate how many parts their computer can handle, or to get an idea of which CPU’s perform best and how much overclocking affects performance, or to just be better informed in their bickering.

    Data is given in the second post; details about how the data are shown and calculated are in the third post; details about testing conditions are in the fourth post; details about the rocket and its stages are in the fifth post.


    You can PM me or leave a link to your results in this thread.

    You can download the rocket from CurseForge, or from the alternate, MediaFire link below:

    http://kerbal.curseforge.com/shareables/220893-cpu-performance-rocket-stock

    http://www.mediafire.com/download/jo..._600_v23_5.zip
    Last edited by DMagic; 1st August 2014 at 19:17. Reason: Add CPU logger plugin

  2. #2
    Capsule Communicator
    Join Date
    Apr 2013
    Location
    Brooklyn, NY
    Posts
    1,760
    Blog Entries
    1
    Results - FPS Charts & Benchmark Comparisons

    Link to high resolution charts.

    Link to All-CPU, high resolution chart.

    To view charts in full resolution click on the icon in the top right corner of the album.

    Some interesting comparisons between CPU types are shown here:



    CPU Benchmark comparison and breakdown by CPU family





    Excel file with all FPS results: https://skydrive.live.com/redir?resi...t=file%2c.xlsx
    Last edited by DMagic; 16th April 2014 at 20:18. Reason: New albums

  3. #3
    Capsule Communicator
    Join Date
    Apr 2013
    Location
    Brooklyn, NY
    Posts
    1,760
    Blog Entries
    1
    Discussion:

    A few things are clear from looking at the labeled CPU comparison chart at the top of the second post. Mobile CPUs tend to underperform based on their benchmark scores. This isn't really surprising, these chips have thermal and power constraints that desktop parts don't. That said, high end mobile CPUs perform very well; the i7 3630QM and the i7 4650U (a 15W ultrabook CPU) put up very good numbers, and both are GPU limited for most of the run, indicating that they still have some headroom left.

    AMD CPUs tend to underperform, every AMD CPU lies below the trendlines. This isn't too surprising really, AMD's selling point recently has generally been price over performance, though I don't have enough AMD results to say much.

    The Core 2 Quad CPUs line up really well. I have a lot of results from these CPUs and they scale very well with clock speed. The differences between them are fairly minor, different cache size for instance, so it's nice to see that different people's results agree so well.

    Intel's recent Core series CPUs all tend to perform very well, the i7 920's really outdo themselves. Overclocking produces a definite improvement, but there seems to be a limit to this. An i7 2600K at 4.7GHz performs only slightly better than a 2500K at 4.0GHz. And if you are looking to upgrade, it seems that Haswell CPUs are a good choice.

    Data Analysis:

    The CPU comparison plot is based on the Passmark score plot from ThePsuedoMonkey. It compares framerates during the initial stage of the launch to both Passmark and Cinebench 11.5 scores, two benchmarks used to give an estimate of CPU power. Framerates are averaged over the first 100 seconds of flight, or the first 2000 frames if frametimes are used.

    Data for Passmark scores come primarily from the passmark database.
    Overclocked values were individually looked up online, and sometimes estimated using results from CPUs with similar overclocked speeds.

    Data for Cinebench scores comes from several sources including http://www.cbscores.com/index.php?sort=rend&order=desc, the Anandtech database and some of their reviews, other values were looked up individually. Notebook scores are primarily from http://www.notebookcheck.net/Mobile-...st.2436.0.html.

    The data file can be accessed on Google Docs, or the updated excel file in the post above:
    https://docs.google.com/file/d/0B3cA...it?usp=sharing

    The data sets below the comparison chart are grouped by processor type. Multiple runs from the same person, using the same settings but at different clockspeeds (or GPU setups in some cases) are denoted with a marker: *, #, $, etc... in the legend.

    The end points for different runs don't necessarily match up. I recommend stopping once the parts separated in stage 6 fall out of physics range. That occurs about 25 seconds of game-time after igniting stage 5. At this point the part count is very low and the frame rate is generally very high (over 60 FPS in most non-mobile cases). The burn time for stages 5 and 3 are fairly long, about 2:10 each, so it's a little dull to watch those stages burn out. You can continue on until you're out of fuel if you want, but in some cases, as you might notice above, I have cutoff the last stages. I've looked at enough of these graphs to know what stage you're in at any given time.

    I did a little work on the frametime graphs. You can see the raw data on page 2 for some of these plots. There are a few differences in my version. I changed the graph to show FPS vs. time in seconds to match up with the rest of the results. And I simply removed those very high speed frames that pop up near the end (less than 8ms, or over 125 FPS). I think these are just runt frames, tiny slivers of a frame that get thrown on screen between two larger frames; these would get removed by V-sync, even on a high speed monitor. Here is a good discussion and demonstration of what runt frames are: http://techreport.com/review/24553/i...apture-tools/9. These sometimes occur in multi-GPU setups, but it looks like they might be happening here, too. Detailed frametime analysis requires a much more complicated setup than just using FRAPS (both hardware and software), and that is discussed in the link above, too, so I'm not going to worry too much about those results. Those very slow frames are a different story though, they are real and noticeable as the skips in sound. That is definitely something that needs to be looked at.

    Here is an example of what I did with the frametime data. The raw data are the two columns on the left, these show the start time for each frame. The rest of the data was calculated using the formulas I printed above each column of values. The time was changed to seconds (this is the x-axis), the frametime for each frame was calculated, in ms, then changed to frame rate. I removed the high speed frames by replacing everything above 120 FPS with a blank cell. Then I calculated a 5 frame moving average (this is the y-axis) to make the plots look a little cleaner. Each set of results totals somewhere around 20000 frames, so there is still a huge number of points on each plot.



    If anyone sees any problems with this, or if anyone has suggestions for how to show the results let me know.
    Last edited by DMagic; 1st October 2013 at 10:13. Reason: Explain data analysis

  4. #4
    Capsule Communicator
    Join Date
    Apr 2013
    Location
    Brooklyn, NY
    Posts
    1,760
    Blog Entries
    1
    I have some notes about testing conditions that I’ll give here.

    My data above was collected on a system with these specs:
    Intel i5 3470 @ 3.5GHz during KSP
    AMD 7850 2GB
    8GB DDR3
    Windows 8 x64
    Resolution set at 1680*1050

    Version Comparison:



    The first two tests, in orange and gray, are from 0.21 and 0.20; they are almost identical. There is a very slight performance improvement in 0.21, but that is mostly above 60FPS where it doesn't make much of a difference. 0.22 is in brown; I stopped the test a little earlier, but you can see that there is no improvement in performance, and there actually seems to be a slight decrease in framerates. You can see the bump in performance from 0.22 to 0.23, it's not very big here, but it's noticeable and things appear a lot smoother overall, even if the framerate isn't much higher.

    The two older versions, 0.19 and 0.18, are a different story though. I had to add a few parts (and modify their .cfg files to make them work in the old format) from 0.21 and 0.20 to make sure the rocket worked, but the settings were identical to the previous tests. You can see that there is a huge drop in performance when going back this far. I double checked the settings values and ran the tests a few times, but everything came out the same. I don't remember noticing such a big improvement in performance between 0.19 and 0.20, but it seems obvious here. It could be that the parts I added are affecting something, or possibly the .craft file has some differences that cause problems.

    View Comparison:

    I have changed my video card to an AMD 7750 since some of my earlier tests, which is quite a bit weaker than the 7850. In making this change I have noticed a few things about performance and graphics settings.

    While using the AMD 7850 I tested different view points during take off to measure the effect of rendering the ground and the ocean, which is known to cause performance problems on lower power machines. With the 7850 I noticed little difference in performance regardless of where I was looking.

    These are the results; terrain is set to "high".



    The tests were done as follows:
    Once without moving the camera, just looking straight on from the side the whole time.
    Once while looking up just enough to keep the ground out of view, but not straight up.
    Once while looking nearly straight down.
    And once with a varying view, sometimes looking up, or down, or changing angles.

    Looking down, or keeping the horizon in frame has an effect on performance but it is slight. These tests were done with a Mechjeb unit installed, which causes a decrease in performance (more about that below), but it seems that the terrain doesn't have too much of an effect when using a moderately powerful GPU.

    Now compare that with my results using the AMD 7750. Terrain is set to "default" here. V-sync is off, but the frame cap is set to 60FPS.



    The difference here is obvious:
    In the lowest performing case, shown in blue, I kept the horizon around the center of the frame for the entire ascent.
    For the next test, shown in orange, I didn't move the camera at all, it is looking straight on from the side.
    For the test shown in grey I looked slightly up, just enough to keep the horizon out of frame.
    In the last test, in green, I used the method described in this thread to alter the ocean terrain detail. Then I kept the horizon in the center of frame, the same as in the first, blue, test.

    This demonstrates the importance of keeping the horizon out of view during the test, especially if you have a weaker, or integrated, GPU. Changing the ocean terrain setting helps a lot too. Above 160km in altitude the performance hit from the terrain disappears and the frame rate shoots up to its limit.

    Another interesting observation was the effect that MechJeb can have on performance. I had heard about MJ affecting frame rates before, but I never paid much attention until now. Here are the data for launches with and without MJ:



    You can see that there is a significant performance hit. I found out that this is due mostly to the Delta-V window being open. Closing that can significantly improve your frame rate, which is something to keep in mind for your own launches. This also demonstrates why I don’t want anyone adding anything to the rocket, just launch it with stock parts. I used MJ to design and test the rocket, and for detailed part counts, but there is no reason to use it during data gathering.
    Last edited by DMagic; 12th January 2014 at 16:40. Reason: Added more about comparison between versions

  5. #5
    Capsule Communicator
    Join Date
    Apr 2013
    Location
    Brooklyn, NY
    Posts
    1,760
    Blog Entries
    1
    Here are some more specific details about the rocket itself. The two pictures below show the various stages and their part counts.





    The rocket itself is fairly simple, it just has lots of extraneous parts, like the nose cones and parachutes on everything. There is little fuel cross-feeding and no asparagus staging. The lighter radial stages should separate without any issues, and the sepratrons on the heavier boosters should keep them from crashing back into the rocket.

    The only slightly unstable stages are those where only the central core remains from the lowest stage (stage 11 and 8 in the pictures above). This causes a bit of wobbling, but it shouldn’t last long. There are a lot of RCS thrusters and plenty of mono-fuel, so you can activate those if you run into any issues keeping the rocket straight.

    The in-game flight time is a little under 4 minutes, while the real time for me is a bit below 5 minutes.

    Here is a detailed table of the stages, their part count, strut count, the number of engines firing, and the number of sepratrons used:



    The rocket should be stable, and I’ve launched it in this configuration about 10 times in 0.23. I haven’t had a single failure with the current setup. If you have any issues just try launching again, and if something is repeatedly going wrong I can take a look at it and try to fix it.

    And just in case anyone was wondering, yes, you can land on the Mun and return.





    Last edited by DMagic; 12th January 2014 at 16:14. Reason: Updates for KSP version 0.23

  6. #6
    DMagic,

    I think this is a wonderful idea, and I'm surprised that a test such as this has not been performed before. Once I get home, I'll be sure to benchmark a few runs and post my results. I can tell you my system specs right now:
    AMD 1090T @ 3.6GHz (OC)
    AMD 6870 2GB (x2, crossfire)
    8GB DDR3
    Windows 7 x64
    Resolution set at 1920*1080

    I'll run a test of each permutation (OC and not OC, CF and not CF, dual monitors and single monitor), so that'll provide eight data points to start you off.

    Digger412

  7. #7
    Interessting.
    I'll try with my computer :
    • MOBO : Asus P8Z77-V
    • CPU : Intel Core i7 3770K
    • RAM : Kingston DDR3 PC15000 HyperX BEAST - 32Go
    • GPU : Asus GeForce GTX 670 DCU II - 2Go
    • HDD : Seagate Barracuda 7200.12 - 1To
    • OS : Windows 8 Pro

    XaT

  8. #8
    Grumpy Spacehippie regex's Avatar
    Join Date
    Jun 2013
    Location
    Eugene, Oregon
    Posts
    3,732
    Blog Entries
    1
    Quote Originally Posted by DMagic View Post
    I know you're all just secretly dying to help me out with this database. Don't let that wall of text and data scare you off.
    I'll try and run the tests sometime tonight or tomorrow using a mobile i3, I can also run it on an older Core2 Duo if you want but that may take a few days because I've been lazy about the old computer.

  9. #9
    Initiating roll program SkyHook's Avatar
    Join Date
    Jul 2012
    Location
    Michigan
    Posts
    1,040
    Blog Entries
    1
    I am definitely in, but can I do this with dxtory? I would rather not download more software, but if I have to I will, this is a great project.

  10. #10
    Capsule Communicator
    Join Date
    Apr 2013
    Location
    Brooklyn, NY
    Posts
    1,760
    Blog Entries
    1
    Quote Originally Posted by Digger412 View Post
    DMagic,

    I think this is a wonderful idea, and I'm surprised that a test such as this has not been performed before. Once I get home, I'll be sure to benchmark a few runs and post my results. I can tell you my system specs right now:
    I'm a bit surprised too, I've never seen any kind of real frame rata data. And thanks for trying it in so many different configurations, that helps a lot.


    Quote Originally Posted by SkyHook View Post
    I am definitely in, but can I do this with dxtory? I would rather not download more software, but if I have to I will, this is a great project.
    Does Dxtory have any kind of frame rate monitoring function, I can't be sure from looking at it online? Anything that outputs a file with the frame rate captured at regular intervals should work fine. Fraps just puts out a file with frame rate numbers, but other programs put out frame rate and the time stamp for each recording. If Dxtory does either of those it should work fine.

    I think MSI Afterburner also has a frame rate recording function, and there may be other programs, too.

    You could also just watch the frame rate and write it down during the launch. It's pretty easy, at least in the earlier stages, to eyeball the average frame rate. And the shift as parts move out of physics distance is pretty obvious, even if you aren't looking down to see them.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •