beowulf clusters on os x?

[josh]

i'm pretty new to macs and unix ,done a little programming on both, but i understand that a beowulf cluster can be run on os X...
my question is...can beowulf clusters be configured so that programs like photoshop would run more efficiently (say hypothetically i have a 10 other macs lying around with os x on them).. or is setting up a cluster only for specific applications programmed with the cluster in mind.

[tsugaru]

I know that some apps support network rendering in OS X but I don't know if you can make a Beowulf for general host CPU offloading regardless if the support is built-in for said application.

That's a good question actually. I'm keeping an eye on this thread.

[bagpuss]

There are various types of cluster based technology out there.

Generally, you need to compile/write applications so that they may take specific advantage of the cluster.

PVM/MPM based clusters (ie. beowulf):
As far as i know, you need to configure specific applications to use the cluster's various resources (eg. another node of the cluster).

Once a process is running on another cluster node, it becomes expensive to migrate it to another node.

As you run the application, I guess it would be feasible to set things up so that forked processes can migrate to other cluster nodes. However, you would need to work out how the processes communicated (shared memory/data is very IO intensive and so expensive in a cluster - as is migrating loads of processes = code + data space). All in all, this would require quite a bit of programming.

openMOSIX based clusters:
Very feasible, but these are only (currently) available for PC based clusters (due to the large amounts of processor dependant kernal hacking - the porting effort should have "improved" with recent versions since it now supports two different CPU architectures).

I'm not sure if code needs to be purpose written here, but would be surprised if this weren't the case.

Java based clusters:
Harness (http://www.mathcs.emory.edu/harness/) is a Java based (ie. current research) cluster software. It allows for Java applications to auto balance etc. Everytime a thread is created, it can migrate to another cluster node. Communication between threads is done using RMI.

Code still needs to be written to take advantage of the cluster.

Hope that helps,

Carl.

[hayne]

Originally posted by josh
is setting up a cluster only for specific applications programmed with the cluster in mind. Short answer: Yes - applications need to be specifically programmed for a Beowulf cluster.
Long answer: see the Beowulf FAQ:
http://www.canonical.org/~kragen/beowulf-faq.txt

[josh]

thanks hayne for the link...
i really figured that programs would have to be written for parallel computing, but i thought i'd ask... it would be interesting if someone could write software that could evenly distribute the cpu load regardless of what application was running.. maybe someone out there is working on it... i just envision a day whan i can plug my tired old ibook into 100 G5's in a cluster and see GOD!.

josh

[tsugaru]

Maybe Apple has something for the next OS revision 10.4. Automated beowulf thread distribution (I hope I said that right.)

[webmonkey]

Originally posted by josh
i just envision a day whan i can plug my tired old ibook into 100 G5's in a cluster and see GOD!.


That would be very nice! With wireless networking only just taking off, and Rendezvous technology now common in higher-end apps, I think you'll be seeing GOD pretty soon.

Apple's Shake 3 compositing software already uses Rendezvous to hunt down spare machines to borrow processing time for rendering....

I appreciate this isn't quite the same as a Beowulf cluster, but it's not a million miles away. Sorry if this is all off-topic!