Caelinux cluster

Hi
actually, CAELinux already contains several codes compiled with OpenMPI like Code Aster 10.1, Code Saturne 2rc1, OpenFOAM 1.7, Elmer FEM 5.5, Gerris flow solver and Impact.

These codes should all be capable to run on multiple machines, but they all have different procedure to submit a job so you will need to read each manual carefully.

However, CAELinux is ready to build a cluster, you just need to configure it properly. By default, ssh is already configured correctly for local connections without password. To build your cluster, you mostly need to extend this config to have global passwordless connexions with ssh.

So, to build a basic cluster with CAELinux, you would mostly need a bunch of PC with a gigabit ethernet switch and the following procedure:
1. install CAELinux on each node, give different hostnames (node0,node1,...), but same login/password!

2. setup static ip adresses for your nodes and /etc/hosts to list the ip/hostnames so that each node can resolv the hostname of the other nodes

3. setup ssh connexion without password between each node :
a. generate the list of known host keys, on 1st node (repeat for all nodes):
[code:1]
ssh-keyscan -t dsa,rsa node1 >> ~/.ssh/known_hosts
ssh-keyscan -t dsa,rsa node2 >> ~/.ssh/known_hosts[/code:1]

b. copy the folder /home/youruser/.ssh of the 1st node to all other nodes with: (on node0)
[code:1]scp -r ~/.ssh node1:~
scp -r ~/.ssh node2:~[/code:1]

c. test: ssh node1 should allow you to connect without password/confirmation..

4. you may want to mount a shared folder within your cluster. A simple system could be to mount on each node a common directory stored on node0 using sshfs. On node0:
[code:1]mkdir ~/shared
ssh node1 "mkdir ~/shared; sshfs node0:~/shared ~/shared"
ssh node2 "mkdir ~/shared; sshfs node0:~/shared ~/shared"
... [/code:1]
(sshfs is not restored after rebooting, so you will need to repeat this step or modify /etc/fstab..)

Then, read the manuals of the code you want to run over the cluster and see which config files must be updated and how to launch MPI jobs. For Aster, I know that you need to edit /opt/aster101/etc/codeaster/aster-mpihosts and maybe /opt/aster101/etc/codeaster/asrun

I have not tested all these possibilities yet, but I have built several small clusters like this.

Let us know about your experience as I am pretty sure a lot of people are interested

Hi JMB

actually NFS is a better choice for systemwide network sharing.
SSHFS is different as it can be mounted by simple users, so it depends on the use case.

If you need permanent shared folders, NFS is the best choice. If you just want a "quick and dirty" network share for a single parallel run sshfs is more flexible and does not require admin rights.

Performance of NFS is probably also much higher than sshfs.

Let us posted about your progress