Getting Started Guide

Getting Started Guide - HPC
1 of 7
https://wiki.adelaide.edu.au/hpc/index.php/Getting_Started_Guide
Getting Started Guide

From HPC
Contents
1 Logging In
1.1 Windows Operating System
1.1.1 Cygwin/Windows
1.2 Linux and OS X
2 Transferring files to Phoenix
2.1 Using a file transfer client
2.2 Using terminal commands
3 Interacting with the Phoenix System: Mastering Linux Basics
4 Loading software packages
4.1 For advance users: To install your own packages
5 Preparing to submit a job
5.1 Creating a simple job script
5.2 Creating a MPI Job Script
5.3 Debiting compute time: Multiple associations
6 HPC user guides - Selecting a Job Queue
7 Submitting A Job
8 Monitoring The Queue
9 Canceling a Job
Logging In
Access is provided by remote login using the secure shell (ssh) protocol.
To login you need to have the appropriate client software installed on your desktop or laptop computer.
Windows Operating System

PuTTY is the recommended ssh client software for windows. It is a free open source.
1. First, you can use this link (http://the.earth.li/~sgtatham/putty/latest/x86/putty.exe) to download the
Putty program.
2. Once you have downloaded PuTTY, you can follow our PuTTY setup guide to configure the PuTTY
software.
Cygwin/Windows
If you have cygwin installed, then you can open a cygwin terminal and then follow the Linux and Mac
section. If you have never heard of cygwin, you can safely ignore this note.
Linux and OS X
6/07/2016 10:01 AM
2 of 7
Linux and OS X both include an ssh client by default.

1. Open the Terminal application to access the command line interface.
2. To log in to the Phoenix cluster use the ssh command with your University of Adelaide credentials,
e.g.
ssh aXXXXXXX@phoenix.adelaide.edu.au
where you replace the X's with your UofA ID number and when prompted enter your UofA password.
3. To change shell environment permanently, use 'rcshell -s /bin/bash' to assign $SHELL with
/bin/bash.
Transferring files to Phoenix

Before you run your application, you will need to upload your data and/or program code from your
computer to your directory on the Phoenix system. Fortunately, this step is very easy and there are a number
of ways to do this.
Using a file transfer client
For Windows users, WinSCP (https://winscp.net/eng/docs/introduction) is an ideal place
to start. Download the WinSCP program here (https://winscp.net/download
/winscp575setup.exe)
If you need help to set up WinSCP, you may find this step by step WinSCP guide useful.
For Mac users, Cyberduck (https://cyberduck.io) is an option, you can download
Cyberduck here (https://update.cyberduck.io/Cyberduck-4.7.2.zip). Use SFTP to establish
the connection. Fetch (http://fetchsoftworks.com/) is another alternative, however, you
will have to register as a educational user to gain licence (http://fetchsoftworks.com/fetch
/free). You can download Fetch here (https://fetchsoftworks.com/fetch/download
/Fetch_5.7.5.dmg).
If you need help to set up Cyberduck or Fetch, you may find this step by step Cyberduck
setup guide or this step by step Fetch setup guide useful.
Using terminal commands
Using the command line interface may seem like a challenge to start with, but becomes easy with practice.
The two common commands for transferring files using the terminal are scp and sftp.
scp uses a similar syntax to the local cp command by specifying the file to copy and the
destination:
scp myfile.txt aXXXXXXX@phoenix.adelaide.edu.au:Path/To/MyPhoenixFolder
Remote file locations are specified by prepending the file path with the user and hostname as in
the example above. For further details you may wish to refer to this scp command Tutorial
(https://www.garron.me/en/articles/scp.html).
sftp (secure ftp) provides a similar interface to the ftp command. First, navigate to the
local directory where your files reside. You can then initiate an sftp session to phoenix
with the following command:
sftp aXXXXXXX@phoenix.adelaide.edu.au
6/07/2016 10:01 AM
3 of 7
Once the sftp session has started, you can use put and get to upload and download files to/from
the remote computer. There are other commands available using the sftp protocol, to learn more
you can look at this sftp tutorial (https://www.garron.me/en/articles/sftp.html).
Interacting with the Phoenix System: Mastering Linux Basics

Like most HPC systems, Phoenix uses Linux as its operating system. Interacting with the system is easier if
you are familiar with the basics of the Linux command line interface. As a starting point to learn about
Linux basics and discover the most useful commands for using Phoenix, you can refer to our Linux for HPC
guide.
Loading software packages

Software packages on Phoenix are organised in modules, some of the currently available packages and
documentations can be found here.
In most cases, your required software is not loaded by default on the Phoenix system. After logging in, you
will need to load your required software before you can perform any calculations. Phoenix uses the module
system to manage the software environment. To see a list of the available software, use the module avail
command as in the example below. If you can not find your required software under this list, there is a good
chance we can make it available to you, contact us via email (mailto:hpcsupport@adelaide.edu.au) to make
a software installation request.
$ module avail
----------------------------------------------------- /usr/share/Modules/modulefiles -------------------------------dot

module-git module-info modules
null
use.own
------------------------------------------------------------ /etc/modulefiles --------------------------------------cuda/6.0

cuda/7.0
gnu-parallel/20150322 intelmpi/5.0.3.048
openmpi/gnu/1.8.4
subread
cuda/6.5
gcc/4.8.4
intel/13.1.3.174
matlab/2014a
openmpi/intel/1.8.1
To load a software package, use the module load command:

$ module load cuda/6.5
To unload a software package, use the module unload command:

$ module unload gcc/4.8.4
To swap between software packages, use the module swap command:

$ module swap cuda/6.5 cuda/7.0
For advance users: To install your own packages
Preparing to submit a job

6/07/2016 10:01 AM
4 of 7
There are two components required to submit a job for analysis in the Phoenix system.
1. The software you wish to run (and any associated data files)
2. A job script that requests system resources
To ensure a fair and optimized system for all Phoenix users, we use a resource management tool, SLURM,
for job scheduling. In order to submit a job to Phoenix, you must create a SLURM job script and save this
along with the program files, into your directory folder on Phoenix. Below are examples of sample job
scripts called <my_job.sh> for each of the two common job types, namely simple jobs and parallel (MPI)
jobs. For each job type, a downloadable version is provided for you to use. Please configure your job script
according to one of following that best suits your requirements.
Creating a simple job script

A job script is a text file that specifies the resources your code needs to run. The job scheduler then uses
these resources to determine when to to run your job. Let's have a look at a simple job script example for
some sequential code (that only runs on 1 CPU core):
#!/bin/bash
#SBATCH -p batch
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --time=01:00:00
#SBATCH --mem=32GB
#
#
#
#
#
# Executing script (Example here is sequential script)

./my_sequential_program
# your software with any arguments
partition (this is the queue your job will be added

number of nodes (use a single node)
number of cores (sequential job uses 1 processor)
time allocation, which has the format (D-HH:MM:SS),
memory pool for all cores (here set to 32 GB)
We'll begin by explaining the purpose of each line of the script example:
The header line #!/bin/bash simply tells the scheduler which shell language is going to be used to
interpret the script. The default shell on Phoenix is bash.
The next set of lines all begin with the prefix #SBATCH. This prefix is used to indicate that we are
specifying a resource request for the scheduler.
The scheduler divides the cluster workload into partitions, or work queues. Different partitions
are used for different types of compute job. Each compute job must select a partition with the -p
option. To learn more about the different partitions available on Phoenix, see <reference>.
The Phoenix cluster is a collection of compute nodes, where each node has multiple CPU cores.
Each job must specify the CPU resources required by using the -N option to request nodes and
the -n to request the number of cores per node required. See <reference>
Each compute job needs to specify an estimate of the amount of time it needs to complete. This
is commonly referred to as the walltime, specified with the --time=HH:MM:SS option. The
estimated walltime needs to be larger than the actual time needed by the job, otherwise the
scheduler will terminate the job for exceeding its requested time.
Dedicated memory (RAM) is allocated for each job when it runs, and the amount of memory
required per node must be specified with the --mem option.
A simple job is one in which the computational process is sequential and is carried out by a single node.
(Note: If your program file does not use MPI or MPI enabled libraries, your job belongs to this category.)
Depending on your computational needs, you may need to use either CPU or GPU-accelerated computing
nodes. This post provides some insights about the differences between CPU and GPU-accelerated
computing. If you need further support with this, please contact the team to discuss.
Below is a sample job script for simple CPU jobs. You will need to create an .sh file in your directory on
6/07/2016 10:01 AM
5 of 7
Phoenix, and can copy and paste the script below into that file. You can also download this job script file,
transfer to your directory on Phoenix, and edit as required. Please remember you must then configure the job
script to your needs. The most common fields that need modification are the number of nodes and cores you
wish to use, the duration of time for which you wish to run the job, and the email address to which
notifications should be sent (i.e. your email address).
#!/bin/bash
#SBATCH -p batch
#SBATCH -N 1
#SBATCH -n 2
#SBATCH --mem=32GB
#
#
#
#
#
# Notification configuration
#SBATCH --mail-type=END
#SBATCH --mail-type=FAIL
#SBATCH --mail-user=firstname.lastname@adelaide.edu.au
# Type of email notifications will be sent (here set

# Email to which notification will be sent
partition (this is the queue your job will be added

number of nodes (due to the nature of sequential pr
number of cores (here uses 2)
time allocation, which has the format (D-HH:MM), he
# Executing script (Example here is sequential script and you have to select suitable compiler for your case.)
bash ./my_program.sh
# bash script used here for demonstration purpose, yo
For simple GPU jobs, the following example job script can be copied and pasted into a new .sh file in your
Phoenix directory, or you can also download this job script file , transfer to Phoenix and edit as needed.
#!/bin/bash
# Configure the resources required
#SBATCH -p batch
#SBATCH -n 8
#SBATCH --gres=gpu:4
#SBATCH --mem=16GB
# partition (this is the queue your job will be added

# number of cores (here uses 8, up to 32 cores are pe
# time allocation, which has the format (D-HH:MM), he
# generic resource required (here requires 4 gpu core
# memory pool for all cores (here set to 16 GB)
# Configure notifications
#SBATCH --mail-user=my_email@adelaide.edu.au

# Execute your script (due to sequential nature, please select proper compiler as your script corresponds to)
bash ./my_program.sh
# bash script used here for demonstration purpose, yo
Creating a MPI Job Script

A parallel (MPI) job is one that harnesses the computational power of multiple nodes, which are networked
together and will perform related calculations or processes simultaneously. This can allow highly complex
computational processes to be performed in much shorter time-frames. To enable parallel computing, the
program you use will need to be MPI enabled or incorporate MPI enabled library. If you do need to run a
parallel job on CPUs, following job script is an example, you can download this script here.
#!/bin/bash
#SBATCH -p batch
#SBATCH -N 2
#SBATCH -n 64
#SBATCH --mem=32GB
#
#
#
#
#
partition (this is the queue your job will be added to)

number of nodes (here uses 2)
number of cores (here 64 cores requested)
time allocation, which has the format (D-HH:MM), here set to 1 hou
mpirun -np 64 ./my_program
For jobs that use GPU accelerators, <my_job.sh> will look like something like the example below. You can
also obtain a copy here.
#!/bin/bash
6/07/2016 10:01 AM
6 of 7
#SBATCH
#SBATCH
#SBATCH
#SBATCH
#SBATCH
-p batch
-n 2
--time=01:00:00
--gres=gpu:1
--mem=16GB
# partition (this is the queue your job will be added to)

# number of cores (here 2 cores requested)
# time allocation, which has the format (D-HH:MM), here set to 1 hou
# generic resource required (here requires 1 GPU)
# memory pool for all cores (here set to 16 GB)
mpirun -np 8 ./my_program
More commonly used options, which can be added to #SBATCH lines includes
#SBATCH --mail-user=my_email@adelaide.edu.au
# Type of email notifications will be sent (here set to END, which m

# Type of email notifications will be sent (here set to FAIL, which
SLURM is very powerful and allows detailed tailoring to fit your specific needs. If you want to explore all
available SLURM parameters available, simply type following in the command line:
man sbatch
Debiting compute time: Multiple associations

The -A association argument specifies the association from which you wish the compute time to be be
debited. Note that you normally only need to specify an association if you have access to multiple
allocations, otherwise the scheduler will debit the resources used from your default association.
HPC user guides - Selecting a Job Queue

Standard compute jobs should select the batch queue. For further details and examples, please refer to the
job requirement guide.
Submitting A Job
To submit a job script to the queue use the sbatch command:
$ sbatch my_job.sh
If your job script requires additional variables you can define these with the --export option to sbatch:
$ sbatch --export=ALL,var1=val1,var2=val2 my_job.sh
Be sure to include the ALL option to --export to ensure your job runs correctly.
Monitoring The Queue

You can view your job's progress through the queue with the squeue command:
$ squeue
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
2916 batch my_job. a1234567 PD 0:00 1 (Resources)
2915 batch my_job. a1234567 R 01:03 2 phoenixcs[1-2]
2914 batch my_job. a1234567 R 00:21 1 phoenixg1
6/07/2016 10:01 AM
7 of 7
The fifth column gives the status of the job as follows: R - running, PD - Pending, F - Failed, ST - Stopped,
TO - Timeout
Running squeue without arguments will list all currently running jobs. However if the list displayed is too
long for you to easily locate your job, you can limit the search to your own jobs by using -u argument like
this
squeue -u aXXXXXXX
where aXXXXXXX is your UofA ID number
Canceling a Job
To cancel a job you own, use the sbatch command followed by the slurm job ID:
$ scancel 2914
To cancel all jobs you own in a particular queue, use the -p argument:
$ scancel -p batch
Retrieved from "https://wiki.adelaide.edu.au/hpc/index.php?title=Getting_Started_Guide&oldid=2100"

This page was last modified on 16 June 2016, at 07:01.
This page has been accessed 2,393 times.
6/07/2016 10:01 AM

Getting Started Guide - HPC Resources

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Getting Started Guide - HPC Resources

Uploaded by

Copyright:

Available Formats

Getting Started Guide - HPC

Windows Operating System