KSP on a Linux cluster ?

[Roxette]

I was looking at a currently idle rackload of quad Xeon servers the other day and wondering if anyone has tried running KSP on a high performance cluster? The physics calculations should fly, but I suspect the bottleneck of the graphics interface would be such a limitation that it may not be worthwhile.

[SSSPutnik]

Curious as to this as well.

[J.Random]

KSP on a high performance cluster? The physics calculations should fly

Who said that? You can't just throw an app into clustered environment and say "now I have a clustered app which is running on 100 CPU cores", that's simply stupid and, when executed in production environment, usually leads to foobars. Application should support multithreaded calculations even to benefit from multiple cores on a single chassis. To even start to work within a cluster, it has to support some sort of MapReduce algorythm. In its current state, KSP won't benefit even from multiple cores of a single chassis. It surely won't benefit from being launched in the cluster. Please don't touch expensive equipment which you have somehow got access to.

Edited December 7, 2013 by J.Random

[Murph]

I agree with what's been said already, in general. Cluster computing is not something you just slide into place under any old software and get any benefit. Applications have to be specifically designed and implemented with a specific style of clustering in mind. Some applications are even more or less impossible to spread over a HPC cluster in a useful manner, if their data set and calculations can not be suitably sliced into convenient subsets. Nearly any application can run on an HA cluster in active/standby mode, and only some a suitable for a HA cluster in active/active mode, but HA clustering is for 24x365 uptime, not for performance. You were looking at a HPC cluster, from the description you gave.

KSP is not designed to, and can't use any more than the hardware of a single system. You do get some benefit from having more than a single core, however, despite KSP not yet being multi-threaded, in that the other core(s) can process background system stuff, any other applications that are still running, etc  ensuring that KSP gets as close to max CPU time as possible.

[SSSPutnik]

I agree, was more wishful thinking, you may get somewhere with a hypervisor.

[J.Random]

You do get some benefit from having more than a single core, however, despite KSP not yet being multi-threaded, in that the other core(s) can process background system stuff, any other applications that are still running, etc  ensuring that KSP gets as close to max CPU time as possible.

While it's generally agreed that binding KSP to one core and everything else to another one will give you some benefit, I'm not sure said benefit will be better than, say, using HT-enabled single core CPU (which may be clocked way higher than multicore system).

I agree, was more wishful thinking, you may get somewhere with a hypervisor.

I'm not sure what you're going to do with hypervisor, or even what you're calling "hypervisor" in this context - if you're talking about HW virtualization then no, you won't benefit from using it. It can split CPU resources, not combine multiple cores into single "supercore", afaik.

[SSSPutnik]

VMware will present the CPUs as a single core processor.

Performance wise, I have not tested. I doubt you with get more flops than running on bare metal.

[J.Random]

VMware will present the CPUs as a single core processor.

Erm, no. 1 single-core vCPU will map into one physical core. Unless I don't know something, which is quite possible, but even then I'm not sure how this technology is supposed to work. Common sense tells me that if guest OS is seeing a single CPU then it will use single instruction queue, and I can't see how you could forcibly split it and feed it into multiple physical cores simultaneously.

[SSSPutnik]

Yeah agreed.