Getting Familiar with Amazon EC2

Due: April 30 2017.

The main goal of this hands-on homework is to become faimilar with the Amazon elastic compute cloud (EC2) platform.

The exercise involves a series of steps. Please follow the instructions carefully to execute these steps. Each step provides familiarity with a certain functionality provided
by EC2.

You will need security credentials (certificates and keys) to carry out the assignment.

Please turn in a short (1-2 page) report with your observations and output from some of the key steps in this assignment. You will also required to provide an estimate
(approximate estimates are OK) on how much your EC2 operations will cost the instructor. Use Amazon's pricing policy and monitoring data to compute your estimates.

In your report, you should clearly mention which EC2 api commands you tried, and All the AMI and instance IDs that you created.

Please remember that we are paying actual money to Amazon based on your usage -- so be very careful with the resources you use, and if you are unclear on
any aspect, ask us for clarifications.

WARNING: All users in this class will be placed in the same security group - which means that you have privileges to terminate a server started by another fellow student.
Please do not abuse these privileges. Update: Amazon has added safety restrictions to student accounts and this warning is no longer applicable.

We may add further explanations on some of these steps to clarify - we will notify the class by email if we do so.

Step 0: Obtain EC2 credentials

Register for an AWS starter account at https://aws.amazon.com/education/awseducate/.
The account activation email contains link the qwiklabs page. Click on the "AWS Account" tab. For your reference, the qwiklabs link is
https://www.awseducate.com/student/ .
Start "Lab 1". We will not be actually doing the lab, but this step is necessary to get AWS credentials. Doing this generates the access and secret keys in the bottom
right part of the window. Copy these to in your bash ~/.bashrc (see Step 1 below).
Click on "Open Console" on left. This will open the full AWS service GUI in another window.
Goto top-left services tab -> EC2 .
On the top-right corner, note and select the geographical region. "N. Virginia" is US-east-1, and is the default region for command-line API tools. If the region is
something else (like Oregon), please ensure that either you select N.Virgina OR specify the appropriate region (like US-west-1) in all your ec2 API calls.
Then goto "Key Pairs" in network & security.
Click on key-pairs in the EC2 part of the console. A qwik labs ssh keypair is already created for you. You will need to create (say, 677kp) and download another
keypair. This is your RSA private key that will enable you to ssh into your ec2 instances.

Step 1: Install EC2 tools

You can download and install the tools in the EDlab machines, or on your local machine

Download and extract the ec2-tools EC2 API Tools

It is convenient to define some environment variables. EC2_HOME represents where your ec2-api-tools are located. We will also use this location for storing your keys
(obtained in step 0). Remember to source ~/.bashrc for these exports to take effect.
export JAVA_HOME=/usr/lib/jvm/default-java
export EC2_HOME=Location of ec2-api-tools
export AWS_ACCESS_KEY=your-aws-access-key-id # Copied from qwiklabs tab
export AWS_SECRET_KEY=your-aws-secret-key # Copied from qwiklabs tab
export PATH=$PATH:$EC2_HOME

Each of you will be have four files from step 0. Let us assume that you are group1. Copy these files to your EC2_HOME
1. Test your environment by running ec2-describe-regions command
$ ec2-describe-regions
REGION eu-west-1 ec2.eu-west-1.amazonaws.com
REGION us-east-1 ec2.us-east-1.amazonaws.com
REGION ap-northeast-1 ec2.ap-northeast-1.amazonaws.com
REGION us-west-1 ec2.us-west-1.amazonaws.com
REGION ap-southeast-1 ec2.ap-southeast-1.amazonaws.com

Step 2: Create an Instance and record its approximate starting time

In this step, you will start up a new Linux server on the EC2 cloud. A server is refered to as an EC2 instance. To start a server, you need to specify a machine image (think
of it as a boot disk). Machine images are called AMI (amazon machine image) in EC2 terminology.

One can create an EC2 instance by specifying an ami-id. But before you create an instance you need to know what kind of ami to use. This is how you would do it

Amazon provides default Linux machine images. Depending on the region, pick the appropriate linux or ubuntu AMI from https://aws.amazon.com/amazon-linux-ami/.
Please select "HVM" AMIs only. For example, use ami-22ce4934 for US-east-1 region, and ami-9e247efe for US-west-1. The next step is to choose an instance type,
for which we pick the t2.medium type.
Please do not create bigger instances as every time an instance is created the account is charged (even if it is immediately terminated)
You can create an instance of this AMI as follows: In this example the key (-k 677pk).
ec2-run-instances ami-22ce4934 -t t2.medium -k 677kp

This will return the instance id, something like i-487587d5. The instance id is the identifier for your instance, and will be required for most of the EC2 commands. After
a few seconds, the instance should have started, so check the status of this instance.
ec2-describe-instances i-487587d5.

This gives various details about the instance type, who is running it, and the address where you can access the instance, something like ec2-54-197-107-
158.compute-1.amazonaws.com .
Record the approximate time it took to create the instance (seconds? minutes?)

Step 3: ssh into the Newly created instance

ssh into the instance using your private key pair and the domain name of your instance
Make sure the 677kp.pem has the right file permissions
chmod 400 677kp.pem

ssh -i /path/to/677kp.pem ec2-user@ec2-xx-xx-xx-xx.compute-1.amazonaws.com

Its possible this may fail/time-out. If that is the case, go to the EC2 console, and go to the "Security Groups" tab on the left. Select the default security group, and
under Actions, select "edit inbound rules". Now in the "Source" column, select "Anywhere". This will allow your instance to be reachable by SSH.

Step 4: Install an application

You can install software on ec2 instances just like on any other linux machine:
yum install -y perl emacs

Step 5: Create a new Image from running instance -- record the time of each operation
Now we can take our customized server and create a new machine image using its disk state. This will allow us to create a new future instance with these customized
applications preinstalled.
Use the ec2-create-image command, and give it the instance-id and a name for your new image. The name should be something like 677img_groupId.
ec2-create-image i-487587d5 -n 677_test_image

If successful, this returns a new AMI, note down this ami-id.

check if the AMI has gotten registered ...
$ ec2-describe-images | grep your-new-ami-id

Step 6: Create an instance of this new image and terminate all your instances -- record time of
each operation
You can now terminate your old instance, since we already have created a machine image with your new changes.
ec2-terminate-instances instance-id

You can approximate the time it takes to terminate by looking at the ec2-describe-instances output and see the status of your instance (running/ shutting-down/
stopped/ terminated, etc)
Now you can start another instance from your customized AMI!
$ ec2-run-instances new-ami-id -k 677kp -t t2.medium

Note that the instance id of this newly created instance is different. Note the new instance id, because the next step is to terminate this new instance as
well. Terminate the instance using the ec2-terminate-instances new-instance-id command as before.

Step 7: Compute the cost of this entire operation

Use Amazon's pricing policy to compute the cost of all operations you performed in this assignment---take into account the time for which your EC2 instances ran,
cost of network I/O, cost of S3 operations etc.

Step 8: Cleaning Up..

Be sure to terminate only your instances; since all projects groups are in the same AWS security group, you can indvertanly terminate someone else's instanes as
well, so be careful with this command.
The last step is to deregister your newly created AMI by
ec2-deregister ami-id-here

Deregistering an AMI does not actually delete it. You need to delete the snapshot associated with the AMI. First find the snapshot by
ec2-describe-snapshots | grep ami-id-here

SNAPSHOT snap-beba4fd9 vol-9ccd304d completed 2016-04-10T19:03:51+0000 100% 101840822718 8 Created by CreateImage(i-487587d5) for ami-9eddcdf4 from vol-9ccd304d

We are finally ready to delete this snapshot. Note the snapshot ID from the previous command and then:
ec2-delete-snapshot snap-beba4fd