CAELinux

  • Increase font size
  • Default font size
  • Decrease font size

CAELinux in the Cloud

E-mail Print PDF

CAELinux on Amazon Elastic Cloud Computing infrastructure

Do you wish to run a large simulation job but you cannot afford to pay for an high-end server with 64gb of RAM? You CFD simulation are taking days on your laptop and you would like to speed-it up using an "on-demand" Cluster?

CAELinux in the Cloud may be the solution you were waiting for !

Using Amazon Elastic Compute Cloud, you can now run as many CAELinux servers, on-demand, with the hardware  that fits your simulation needs (up to 8 cores / 64 GB RAM) and this just by paying the server hours  !

Update: for an up to date procedure please follow this thread on the forum

Description

Amazon Elastic Cloud Computing (EC2, see http://aws.amazon.com/ec2/ ) is the one of the largest virtual machine hosting service and allows to start "on-demand" as many servers as one may require to achieve a given computational task. The service is available to anyone and the execution of the server instance is charged per hour at a very reasonnable price. The main advantage of EC2 for scientific computation is that it allows to run extremelly large jobs on huge servers having up to 64Gb of RAM and 8 cores for a few hours without the need to buy an expensive high-end server hardware.

To benefit from this great flexibility, CAELinux 2010 has just been transfered and adapted to run on Amazon EC2 cloud, so that anyone can run the image on the EC2 server farms on-demand with all the great CFD/FEA softwares pre-installed and ready to run your simulation jobs. Another great advantage of CAELinux in the Cloud is that you do not pay expensive simulation software licences, but just the price of running the hardware, so you can run for example very large CFD jobs spanning for examples 4 nodes with 8 cores each without paying a cent for licences!!

To access the running instance on EC2, you can either use a traditionnal SSH terminal and SFTP to transfer files to / from your workstation. But you can also run CAELinux in the Cloud as a virtual desktop thanks to the pre-installed NX remote desktop server. Using the free NoMachine NX  Client on your workstation, you can thus run a full featured Linux CAE workstation remotelly through a basic internet connexion.

This brand new CAELinux EC2 image is currently released as BETA version and is provided for testing purpose only. The goal of this 1st release is to allow you to test the service and give us some feedback on what needs to be improved and what you would need.

Using CAELinux in the Cloud in Amazon EC2

First of all you will need a working Amazon Web Service (AWS) account to use the Amazon EC2 service. For more information on Amazon AWS please see http://aws.amazon.com

To run a CAELinux instance:

1) Sign in in the AWS management console at http://aws.amazon.com and  enter the EC2 tab,

2) select Region = EU West and move to the "Images AMIs" (Amazon Machine Image) category in the left menu

3)  Select the filters "Public images" and search for the keyword "CAELinux".

You should see the following server image:

AMI ID Source Kernel ID
Architecture Root
ami-360b3b42 008531201741/CAELinux2010.2-20110730 aki-2e4a7f5a x86_64 EBS

4) Right click on the image and choose "Launch instance" to run a server using this image.

Choose the number of instance to run (1 is a good guess ;-) )

Choose the type of server hardware (Large, m1.large, 7.5Gb RAM, 2 cores is a good default value)

Please note that the price per hours depends heavily on the server type see http://aws.amazon.com/ec2/#pricing for more info. Use Spot Instances for the lowest price: http://aws.amazon.com/ec2/spot-instances/

Continue the wizard (leave kernel / ramdisk settings on default) and select or create a "key pair" if necessary.

If not done already, create a Firewall / Security group rule to allow SSH access ( tcp port 22, source 0.0.0.0/0).

Validate the option of the Launch Wizard .

5)  Go to Instances tab: you should see the status of your instance(s). After around 2-3 minutes, the server shouldbe accessible.

To access a running instance :

A running instance has two ip adresses, one for the internal network inside the AWS cloud and a second one that is accessible from anywhere on the internet. These two ip adresses correspond to two DNS names. The one you need to know to access your server should look like   ec2-XX-XX-XXX-XX.eu-west-1.compute.amazonaws.com

Select the running instance and in the lower tab , copy the Public DNS field in the clip board for further use.

a) To access your server using SSH (terminal):

use your favorite SSH client (ssh in linux, putty on windows machines) to connect to your server at the address ec2-XX-XX-XXX-XX.eu-west-1.compute.amazonaws.com (paste the Public DNS address to avoid mistakes). Authentication is based only on your Amazon SSH keypair, so put it in a convenient directory and make sure that you remove read-write permissions for all others: chmod go-rwx mykeypair.pem

then connect using:

ssh -i mykeypair.pem ubuntu@ec2-XX-XX-XXX-XX.eu-west-1.compute.amazonaws.com

once logged in, define a password for the ubuntu user:

passwd

b) then to access your server with NX Client (remote desktop) :

First, install NoMachine NX Client on your machine: see  http://www.nomachine.com/download.php

Then setup a new connection using the NX Connection Wizard, specify

host = ec2-XX-XX-XXX-XX.eu-west-1.compute.amazonaws.com (paste the Public DNS address to avoid mistakes)

connection type = ADSL,

choose the type of Desktop = GNOME (instead of KDE)

choose a desktop size (1024x768 is a minimum)

continue the wizard and then log in using

login : ubuntu

password: "Your custom pasword defined in a)"

After 1 minute max (the 1st time), you should have a window with the remote CAELinux desktop running on Amazon EC2.

c) to transfer files to/from your server :

simply use an SFTP or SCP client to connect to your server . In Windows,you can useFileZilla or WinSCP to connect and transfer files.

On Linux, FileZilla is a good SFTP client. You can also directly use the terminal scp commandor use sshfs to mount a remote folder locally.The simplest option in Linux GNOME, you can directly mount an SSH remote folder using the "Connect to Server" tool.

Whatever tool you use, set the host address to ec2-XX-XX-XXX-XX.eu-west-1.compute.amazonaws.com and use the login = ubuntu and password="your custom password" to connect

Data storage:

By default, the data storage coming with the CAELinux image is only kept during the life of the instance. In other words, if you terminate the instance, the data are lost. To keep the data, you can decide to Stop an instance and restart it later.

Once running, you will notice that the disk space provided by the main system disk is limited (about 2.5 Gb of free space).

For temporary & local storage you can benefit from the local disk of the server. This local disk is accessible at /mnt or at /home/ubuntu/localdisk. Code-Aster has been configured to use the local disk for scratch space, but if you run large simulation, make sure that you configure your simulation tools to put temporary files on /mnt.

If you want persistent storage, the simplest thing to do is to create a new volume in Amazon AWS console and to attach it to your running instance. In this way, you will be able to keep your data even after the EC2 instance is rebooted.

To Stop a CAELinux instance:

To  Stop a running instance and thus stop paying for the service, go to AWS management console, in the EC2 Tab, Instance section. Right click on the Instance you want to terminate and choose "Terminate" in the menu. Approx 1 minute latter, the machine will be turned off and all its content will be lost (reset to the default image). To keep the changes, choose "Stop" instead of Terminate. Be carefull though, EBS Instance Images are not made for data persistence, use attached EBS Volumes for that purpose.
For more details, read the Amazon EC2 documentation at http://aws.amazon.com/ebs/ for more information on Elastic Block Store (EBS)

IMPORTANT NOTE : IF YOU DON'T WANT TO PAY FOR UNUSED HOURS, YOU MUST TERMINATE YOUR INSTANCE FROM AWS MANAGEMENT CONSOLE !!

IMPORTANT NOTE2: If you TERMINATE an instance, all the data it contains is LOST. To keep the data, choose Stop instead. The EC2 EBS image is not meant to be persistent, you should use external EBS volumes for data persistence in the cloud.

User Feedback and Requests

Please, leave us your feedback and comments on the forum.

Don't forget that it is still a beta version.

Cluster in the Cloud

The present CAELinux in the Cloud image contains all the tools to run multiprocessor simulation jobs but is not preconfigured to run jobs on a cluster of EC2 virtual machines.

But, in a near future, CAELinux in the Cloud  will be extended to include a simple application to setup a "CAELinux Cluster In the Cloud" by configuring a set of running CAELinux EC2 instances to run large MPI jobs. This will allow you to run MPI jobs on your virtual cluster using Code-Aster, Code-Saturne, OpenFOAM or Elmer. To help us maintain the development of CAELinux, this Cluster in the Cloud service will be offered as a paid-for annual subscription based service (but at a very reasonnable price).

 

 

Last Updated on Sunday, 09 March 2014 12:07  

Login / Register

Search

Support CAELinux

CAELinux is a non-profit, open-source oriented project that is developed by volunteers.

If you want to support the further developments of CAELinux, you are welcome to make a donation through the following Paypal button.

The CAELinux Team would like to thank you in advance for your support:

Service & support

CAELinux is supported by

www.nrctech.ch