KSP1 Computer Building/Buying Megathread

[Duke23]

I have a question about RAID and similar methods. Basically I need something like a RAID 1 setup but two of them. I want to have two mirrored 4tb disks and two mirrored 2tb disks. So the question is can I accomplish this with a 4 bay hardware RAID enclosure, or do I need two dual enclosures side by side? Or would it be easier (and cheaper) to just get a regular SATA -> eSATA / USB3 enclosure with no RAID controller and use software (what software?) to mirror on the fly, and hopefully only show 2 disks instead of 4 under My Computer.

Sorry if that's a little confusing, I'm just trying to figure out the easiest and most cost effective way to make sure my important data is crash proof. I don't want to overcomplicate things with RAID 5 or anything like that if don't have to. But I may have already overcomplicated things by being so paranoid about data loss. Right now I'm backing up the most important of the important on multiple disks which is inefficient at best and not all inclusive.

Thanks in advance for any insight.

[cantab]

In my view, RAID is not backup. There are many data loss scenarios and it only protects you against one, hardware failure. Rather, RAID (ignoring RAID 0) is about availability - being able to keep the system running despite the hardware failure. Important for a server, maybe useful for a desktop especially if you rely on it for business, but no substitute for real backups.

[Duke23]

Well hardware failure is my biggest concern. In my scenario it won't be running an OS or anything and if the data does become corrupted it probably happened before it made its way to this setup in the first place, and even if it did happen after I might not catch it before I update a backup manually. Once the data goes to this setup it probably won't be modified. I was hoping to have this all pretty much hands free until a disk fails which I could promptly replace. I thought this is what RAID 1 was for, and would it not detect corruption the same way it would detect a bad sector for example and make the appropriate corrections or at least notify me?

I don't know anything about RAID really so I'm trying to learn. What do you consider a "real backup?

[Bilfr3d]

What do you consider a "real backup?

Get a 4Tb External HDD and copy everything onto that. That's the simplest "real backup". If you want a HDD in the computer that just serves as a backup drive, perhaps make a batch file that's copies the files you want to backup, and schedule it to run at a certain type of day, or make it run on shutdown so that it holds the shutdown back until it is complete. Just a few ways of a "real" backup, but there are plenty more ways out there. It just depends on what is easiest for you and if you would rather go for assured backup or a cost-effective backup.

[cantab]

IMHO a good backup should be automatic and should offer some protection against user error and software bugs, as well against hardware failure. Usually this is done by keeping older versions of files.

Ideally it should also protect against malice. That's harder - malware can easily wipe (or in the case of ransomware encrypt) an external hard drive or a network drive. But it can't obviously wipe a drive that's disconnected, and I've yet to hear of any malware trying to wipe online backups.

If you have important financial, legal, or similar documentation on your computer - ie the kind of stuff you would want if your house was destroyed - it would be a good idea to not have your backup in that same house. And if you feel you need your PC, you might want to think about speed and ease of recovery (which is something a full system image can be good for).

Of course you can never totally protect against malice or error, since something could always break your backups and not show itself until much later, but you can do a reasonable job.

For smaller amounts of data online backup is a good approach, provided you're on an unmetred and sufficiently fast internet connection. For larger amounts a pair of (or more) backup drives, periodically rotated between being connected to the PC and stored somewhere safe, is better (but more expensive) than a single one, and a good backup program will handle keeping previous versions of files without duplicating stuff that hasn't changed.

[Camacha]

IMHO a good backup should be automatic and should offer some protection against user error and software bugs, as well against hardware failure. Usually this is done by keeping older versions of files.

I fully agree here. It is clear by now that people, even with the best intentions, slack off after a while and forget to backup when it most counts. I would say that a non automated backup is not a backup.

Ideally it should also protect against malice. That's harder - malware can easily wipe (or in the case of ransomware encrypt) an external hard drive or a network drive. But it can't obviously wipe a drive that's disconnected, and I've yet to hear of any malware trying to wipe online backups.

The thing I am most afraid of is something like a bad RAM stick or other component that causes files to be broken, but not showing it right away. That way you might copy bad data to your backups for weeks or months before noticing and then it is way too late when you need then. I am currently trying to figure out some way of checking on local drive and backups by making hashes and comparing those to themselves and each other, enabling detection of data rot and corruption of the source files, but I am having some trouble with the source disk obviously always changing. That makes it hard to automate corruption detection.

Maybe incremental backups are the answer, but I have always resisted non-flat backupping, as getting to your files easily in case of a disaster is important. If you need some specific setup, procedure or program to extract your files, data loss or having to invest much time becomes a lot more probable.

[steve_v]

I thought this is what RAID 1 was for, and would it not detect corruption the same way it would detect a bad sector for example and make the appropriate corrections or at least notify me?

Not IME. Most consumer RAID cards, GNU/Linux/BSD soft RAID (Never used Windows soft RAID but I assume it's the same) do _no_ consistency checking until you run a 'scrub'. A bad sector can go undetected until it needs to be read - i.e. during a rebuild, this can ruin your day in a big way.

Files aren't checksumed either, so data integrity checks are up to you.

ZFS, BTRFS and ReFS solve these issues

RAID (any level but 0) is more about availability, i.e. you can still use the array while a disk is hot-swapped. As such you still need a backup in case things go really pear shaped.

I once had a dodgy sata connector (I snapped the plastic 'tongue') that would silently corrupt data on a software RAID1

Had fun with RAID5 back in the day too - it's just a matter of time until 2 drives fail within the rebuild window...

Backups are not optional if you value your data.

My current setup, FWIW:

OS array: 2x160GB, 10K RPM as software RAID1 (daily scrub + checksums of OS files)

Data pool: 8x2TB, 7.2K RPM as ZFS RAID-Z6 (checksums on, compression on, daily scrub) with 1x60GB SSD as cache.

Hourly & daily inceremental snapshots of data pool - snaps are virtually free with ZFS, neat.

Daily snapshots of _critical_ ZFS subvolumes + OS array rsync'd (incremental nightly, deleted files purged weekly) to external USB3 4TB drive.

Remaining ~6TB of data pool is not backed up as it's all replaceable.

A bit OP for a home server perhaps, but I like shiny toys.

Edited August 19, 2014 by steve_v

[Camacha]

I tried ReFS for a little while and it worked, mostly. It is a very interesting file format, but not totally mature yet, so I am not sure you would want to use that for your data.

In my case, my virus software could not read the drives formatted this way. It would pretend to scan them, but return a 100% clean result with 0 files scanned. I would gain parity but lose virus scanning, so I reformatted as I feel the latter is more urgent and the former is easily mitigated with a good backup strategy.

Edited August 19, 2014 by Camacha

[steve_v]

Antivirus? what's that?

My one and only Windows install is entirely expendable these days (i.e. the last 10 years or so), has no write access to network resources and simply gets nuked & restored from image if anything nasty* happens to it.

*Most often a botched update, haven't seen a virus / malware in years.

Any Windows boxen I have to work with professionaly are airgapped and/or 'not my problem'

AFAIK the only production ready 'next gen' filesystem is ZFS on BSD or Solaris - the GNU/Linux port is stable, but still slowish.

I'm waiting patiently for the rest to catch up, but with todays disk capacities bit-rot is becoming a problem, likewise rebuild times on traditional RAID.

On the other hand, backups become simpler as large externals get cheaper - tape drives are / were pretty horrible if you ask me.

I even used CDRs for backups for a while, bugger that for a joke, 'oh noes, disk 19 of 47 has a scratch...'

Edited August 19, 2014 by steve_v

[Duke23]

You all bring up good points. I may take a little of all this when I go for this upgrade - I think I still want to do a RAID 1 setup and two sets of disks for the "Oh crap my disk just blew up" protection but also continue using the 3 disks (3tb, 1tb, and 320gb I think) that the data is currently on. It still has room to grow but #1 I'm wasting USB ports and the smaller disk is old but still seems to be in good condition so I leave it alone usually. #2 my 3tb disk has that stupid head parking feature so it is racking up an amount of wear and tear that I'm not going to be comfortable with in about another year of use. I can handle a scheduled integrity check much easier than pulling a disk out of my drawer to sync everything. All this really hit me when my 500gb disk decided to go squirrely on me and I barely got the data transferred off it in one piece before it became unusable.

Does anyone know if I can do my two separate sets of mirrored disks with one RAID controller or should I just use software / two dual enclosures?

Sigh, I do remember the days of using CD-Rs for backup. Or DVD-Rs. Either of which were a perfect recipe for something to go wrong right when you need it. I have always been fond of using flash drives for the really important stuff. They fail too but in my experience they have the courtesy to do it while you're writing data and be pretty obvious about it.

[Camacha]

Even with an air gap you are not impervious to malware - any USB drive can be a hazard too. Admittedly, the risk much smaller, but using Windows without malware protection would not sit well with me.

Of course, my system is not expendable at all and needs to be somewhat reliable production wise - this becomes more and more relevant as my professional duties increase. I take precautions in the case of mishaps, but do my best to prevent mishaps too. The fact that a lot of malware does not show itself before it is too late, shipping off data or encrypting your files, means you cannot be too careful.

I must say I have not had any trouble with Windows updates.

Sigh, I do remember the days of using CD-Rs for backup. Or DVD-Rs. Either of which were a perfect recipe for something to go wrong right when you need it. I have always been fond of using flash drives for the really important stuff. They fail too but in my experience they have the courtesy to do it while you're writing data and be pretty obvious about it.

You might not be aware, but memory cells do have a limited data retention capacity. Storing data on them for long periods of time without supervision and/or power might become a problem because of that. Otherwise, I agree.

[Duke23]

You might not be aware, but memory cells do have a limited data retention capacity. Storing data on them for long periods of time without supervision and/or power might become a problem because of that. Otherwise, I agree.

Yes, but it's "supposed" to be 10 years I believe? I've never gone more than about 2 years without either checking up on or updating one of my flash backups and for the most part those files are already in multiple places to prevent any weirdness. Things like old pictures, old writing stuff etc that's impossible to replace. I haven't had any issues *yet* with leaving them laying around but it's usually one drive that gets upgraded periodically as the files grow or storage space gets cheaper. I think the first one I used for that purpose was something along the lines of 64 or 128mb but now I'm just showing my age a little All this paranoia is pretty much traced back to the fact that I lost a lot of irreplaceable data back in about 2003 when I toasted a hard drive, I was much less careful before that. That reminds me I need to update all that stuff for the last few months worth of stuff... It never ends, I'm just going to go sit in the corner and pull my hair out.

[steve_v]

any USB drive can be a hazard too

Too true, although disabling various default stupidity (like autorun) helps. I cringe every time someone grabs that usb drive from their keyring... where has that been? is it clean? are you sure? etc.

USB ports and floppy drives (yep, still need em sometimes) on mission critical systems are generaly inside locked cabinets, along with the rest of the machine. Usualy with some scary looking electrical stuff just for good measure , this has the added benefit of protection from human interference.

Some of the systems I have to deal with are so old that updates / antivirus is simply not available anymore so the only option is to isolate the hardware. - Try finding a box that can accept ISA cards AND run a supported OS, I dare ya.

You have the key, you plug in the foreign device, you had better know what you're up to.

I will dig out the disk image if you screw it up, but it will cost you lunch ;-)

My preferred solution would be to... NOT run Windows, or DOS for that matter. But what can you do.

I know of a recent incident where someone.. not naming names, but higher up the food chain than me, decided to use a SINGLE micro SD card as the only backup for critical automation programs... you can guess how well that turned out.

Edited August 20, 2014 by steve_v

[Camacha]

My preferred solution would be to... NOT run Windows, or DOS for that matter. But what can you do.

I would agree with you, but for various reasons I am bound to Windows. Simple practicality means I need internet access too, so air gapping has a fairly limited use. I the future I might separate systems into internet non-critical and more critical systems to increase production reliability and robustness and limit downtime, but it simply does not pay to maintain that kind of infrastructure at the moment. Yet, I hopefully add.

Yes, but it's "supposed" to be 10 years I believe?

I remember reading about a theoretic 7 months in a worst case scenario, but I cannot really back that up.

Sorry for the pun.

[Bilfr3d]

Just a quick question here. If I set up 4 identical computers. Each has 6x 3Tb HDD in a RAID 6, a 60Gb SSD for the OS and another 60Gb SSD for cache. Each kit has 2 1GbE ports in a teem, running an iscsi link to a managing computer. Now, on the managing computer, would it be possible to use a software RAID, and make the 4 iscsi links in a single RAID 5? This would be to increase overall uptime, because there would be the hardware RAID on the kits managing the RAID 6 on the individual kits. Then the software RAID on the management computer RAIDing the 4 iscsi links into a single RAID 5, so in the case more than two HDDs fail on a kit before they can be replaced, a hot spare kit kicks in, until you replace the two (or more) HDDs on the failed system. Than that system you just repaired would become the hot-spare in the iscsi link. All in all, would this work?

[cantab]

Possible, probably. Sounds a bit convoluted though, and the gigabit ethernet will probably limit performance.

[Bilfr3d]

Possible, probably. Sounds a bit convoluted though, and the gigabit ethernet will probably limit performance.

Yeah, I was thinking about using fibre cards instead. The dual RAIDs would also slow things down a bit.

[Camacha]

Finally the runt of the Intel *-E line, in this case Haswell-E, is going to make sense. Before it was generally just an overpriced alternative to the middle tier i7 processors, but now they actually get two extra cores for a decent price. I am doing pretty well with my current setup, but it is nice to know that there is an affordable alternative in case I need a little more power to render or simulate. The top tier 8 core 16 thread edition is over priced as always, but even that might make sense in case you earn any money with your rig.

I also understand Ivy-E is going pack to a soldered IHS, one of the major quibbles of every Intel processor since Sandy Bridge. Soldering is just superior to TIM, although I understand it might also be some kind of epoxy like adhesive. The proof is in the pudding, we will see what the temperatures do.

[Endersmens]

Ok guys, so I plan on buying a PC building kit. I'll post the specs of the build i'm gonna get, and I would love it if someone could tell me how powerful the computer will be, and if there's anything I should change on the order. (Im just gonna copy-paste the specs from the website) ideally, I want my computer to be able to pretty much run any game out there, no problem, with reasonable settings, if not really high settings.

Rosewill RANGER-M Dual Fans MicroATX Mini Tower Computer Case (Model:RANGER-M)

ASUS A55BM-E FM2+ / FM2 AMD A55 (Hudson D2) Micro ATX AMD Motherboard (Model:A55BM-E)

Rosewill ARC 550 Continuous 550W@40 C degree ATX12V v2.31 & EPS12V v2.92 SLI Active PFC 80 PLUS BRONZE Certified Power Supply (Model:ARC 550)

AMD A10-5800K Trinity Quad-Core 3.8GHz (4.2GHz Turbo) Socket FM2 100W Desktop APU (CPU + GPU) with DirectX 11 Graphic AMD Radeon HD 7660D AD580KWOHJBOX (Model:AD580KWOHJBOX)

HyperX Fury Black Series 8GB 240-Pin DDR3 SDRAM DDR3 1600 (PC3 12800) Desktop Memory Model HX316C10FB/8 (Model:HX316C10FB/8)

Western Digital WD Blue WD10EZEX 1TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive Bare Drive (Model:WD10EZEX)

LG Internal Super Multi Drive SATA Model - GH24NSB0B (Model:GH24NSB0B)

I will also be adding this graphics card (can someone tell me if its compatible?) :XFX R7-250E-ZNP4 Radeon R7 250E 1GB 128-Bit GDDR5 PCI Express 3.0 x16 Support 3 Monitors Video Card

[Camacha]

and I would love it if someone could tell me how powerful the computer will be

The best way of checking that out is looking up some benchmarks Enter your CPU into Google with the word review or benchmark added, maybe accompanied with the game you are planning to play, and see what comes up. Please make sure to only check out reviews that have actually measured performance, as there are a lot of crap sites out there that will give you numbers based on the specifications. Those are rubbish. Rinse and repeat for the GPU.

I will give you a small indication: the CPU is not a powerhouse, but okay-ish. Not great, not bad. The GPU seems a bit anaemic, I think I would advise save another 70 dollars to buy something that will really serve your purposes and also last a little longer. Also, your CPU contains a chip that is not that much slower, so it seems a bit of a waste to double up. AMD does have technology that allows you to combine the APU with a GPU, but I think you would need a A10 78x0, not the one you currently selected.

It is probably a waste to use a GPU next to a pretty much comparable APU, because you pay twice that way for not much more performance. Either make sure you can double up, or go all-in with a higher end card.

[cantab]

On the RAM, is that a single 8GB stick? If so, then if your motherboard has four RAM slots consider 2x4GB sticks instead, it improves performance. But if there are only two slots I'd stick with the single stick so you can add another at a later date.

On the hard drive, I advise getting an SSD if you can afford it. It makes a big difference to the general responsiveness of your system. If you want to keep the costs down and need plenty of space opt for a hybrid drive, it won't match a proper SSD but will speed things up over a bog-standard mechanical drive.

[Baenki]

Hi. My old PC gave up last week and i need a new one for uni. But it also should be powerful enough to play new games/ edit videos/ animate in flash.

A friend of mine showed me a list he used to configure his own PC.

http://geizhals.de/?cat=WL-140893

It's ~800€/1000$ and there are no hard drives on the list.

I plan to reuse my old drives, maybe my case if the parts fit and will get it assembled by a tech store in my town.

My self enforced budget limit is 1000€/1300$ give or take 10%.

I would like a simple SSD just for my OS and maybe 1 or 2 programs.

What should i spend the rest of my budget on?

I would like a better CPU, GPU or more RAM ir a mix of them.

What do you think makes the most sense.

Edit: Notice: I found that the i7-4790k is only 80€/105$ more.

Edited August 29, 2014 by Baenki

[Camacha]

I think you are making quite good choices already. The Xeon is giving you a lot of performance for a very reasonable price. Of course the i7 is faster, but also a lot more expensive. But if you feel you can spend the money, sure, why not? Of course, if you want to overclock you need a different motherboard too.

I would suggest getting a decent sized SSD, as they tend to get too small after a while. Prices are very reasonable nowadays. A quicker GPU will not do you much good on a 1080P screen, you really need to go tri-screen or high resolution to really put that to good use. More RAM is pretty much useless right now, but DDR3 is on its way out and will likely only become more expensive. It might be a good idea to stock up on 16 GB to be somewhat future proof.

And, as always, ensure you have some decent way of making backups on a second physical drive.

[Endersmens]

The best way of checking that out is looking up some benchmarks Enter your CPU into Google with the word review or benchmark added, maybe accompanied with the game you are planning to play, and see what comes up. Please make sure to only check out reviews that have actually measured performance, as there are a lot of crap sites out there that will give you numbers based on the specifications. Those are rubbish. Rinse and repeat for the GPU.

Noted.

I will give you a small indication: the CPU is not a powerhouse, but okay-ish. Not great, not bad.

Well...right now i'm using a 1.8 GHz dual core AMD processor. I figured a quad 3.9-4.2 GHz would be much much better. And since my computer will run the lower half of games, I figured this CPU would handle most games alright. (KSP for example. Think I could run KSP full graphics and everything with this setup? That's the main goal)

The GPU seems a bit anaemic, I think I would advise save another 70 dollars to buy something that will really serve your purposes and also last a little longer. Also, your CPU contains a chip that is not that much slower, so it seems a bit of a waste to double up. AMD does have technology that allows you to combine the APU with a GPU, but I think you would need a A10 78x0, not the one you currently selected.

Well this is a pre-setup-kit. I didn't hand pick all that. I just picked one that looked good and was in my price range, and figured that would be a good starting point. However i'm totally up for hand picking everything that you suggest. (Price right now is around 370 I believe, I would really really like to stay under 450, 500 at the most)

It is probably a waste to use a GPU next to a pretty much comparable APU, because you pay twice that way for not much more performance. Either make sure you can double up, or go all-in with a higher end card.

I'm not sure i understand, i'm not a huge computer person. Yet.

[steve_v]

Quick question for anyone out there with a Sandybridge-E (or any other recent intel with 'turbo boost'):

Does KSP activate your turbo clocks?

I never seem to get my OC'd 4.3Ghz turbo just running KSP, though it certainly benefits from it, I need to have something else chewing on a core eg. 'stress -c 1' to get it to ramp up.

@Camacha: Good to see you're still hammering away at that backups nail

There are 2 types of computer users: those who do backups and those who have never had a drive fail...

If you've got spare dosh in your budget, get an external or a NAS.

On a side note, a decent SSD is the single biggest subjective performance enhancer I have seen since the 40MHhz FSB hack for 486 boards.

@Baenki, just my 2c, but getting a Xeon on a board that can't OC it seems like a bit of a shame to me. It's been a while since I had one but IME they overclock rather nicely, and KSP just loves CPU cycles.

Edited August 30, 2014 by steve_v
Damn lying motherboard, 44x setting does nothing.