nova/vendor/boto/docs/source/ec2_tut.rst

.. _ec2_tut:

=======================================
An Introduction to boto's EC2 interface
=======================================

This tutorial focuses on the boto interface to the Elastic Compute Cloud
from Amazon Web Services.  This tutorial assumes that you have already
downloaded and installed boto.

Creating a Connection
---------------------
The first step in accessing EC2 is to create a connection to the service.
There are two ways to do this in boto.  The first is:

>>> from boto.ec2.connection import EC2Connection
>>> conn = EC2Connection('<aws access key>', '<aws secret key>')

At this point the variable conn will point to an EC2Connection object.  In
this example, the AWS access key and AWS secret key are passed in to the
method explicitely.  Alternatively, you can set the environment variables:

AWS_ACCESS_KEY_ID - Your AWS Access Key ID
AWS_SECRET_ACCESS_KEY - Your AWS Secret Access Key

and then call the constructor without any arguments, like this:

>>> conn = EC2Connection()

There is also a shortcut function in the boto package, called connect_ec2
that may provide a slightly easier means of creating a connection:

>>> import boto
>>> conn = boto.connect_ec2()

In either case, conn will point to an EC2Connection object which we will
use throughout the remainder of this tutorial.

A Note About Regions
--------------------
The 2008-12-01 version of the EC2 API introduced the idea of Regions.
A Region is geographically distinct and is completely isolated from
other EC2 Regions.  At the time of the launch of the 2008-12-01 API
there were two available regions, us-east-1 and eu-west-1.  Each
Region has it's own service endpoint and therefore would require
it's own EC2Connection object in boto.

The default behavior in boto, as shown above, is to connect you with
the us-east-1 region which is exactly the same as the behavior prior
to the introduction of Regions.

However, if you would like to connect to a region other than us-east-1,
there are a couple of ways to accomplish that.  The first way, is to
as EC2 to provide a list of currently supported regions.  You can do
that using the regions function in the boto.ec2 module:

>>> import boto.ec2
>>> regions = boto.ec2.regions()
>>> regions
[RegionInfo:eu-west-1, RegionInfo:us-east-1]
>>> 

As you can see, a list of available regions is returned.  Each region
is represented by a RegionInfo object.  A RegionInfo object has two
attributes; a name and an endpoint.

>>> eu = regions[0]
>>> eu.name
u'eu-west-1'
>>> eu.endpoint
u'eu-west-1.ec2.amazonaws.com'
>>>

You can easily create a connection to a region by using the connect
method of the RegionInfo object:

>>> conn_eu = eu.connect()
>>> conn_eu
<boto.ec2.connection.EC2Connection instance at 0xccaaa8>
>>> 

The variable conn_eu is now bound to an EC2Connection object connected
to the endpoint of the eu-west-1 region and all operations performed via
that connection and all objects created by that connection will be scoped
to the eu-west-1 region.  You can always tell which region a connection
is associated with by accessing it's region attribute:

>>> conn_eu.region
RegionInfo:eu-west-1
>>>

Supporting EC2 objects such as SecurityGroups, KeyPairs, Addresses,
Volumes, Images and SnapShots are local to a particular region.  So
don't expect to find the security groups you created in the us-east-1
region to be available in the eu-west-1 region.

Some objects in boto, such as SecurityGroup, have a new method called
copy_to_region which will attempt to create a copy of the object in
another region.  For example:

>>> regions
[RegionInfo:eu-west-1, RegionInfo:us-east-1]
>>> conn_us = regions[1].connect()
>>> groups = conn_us.get_all_security_groups()
>>> groups
[SecurityGroup:alfresco, SecurityGroup:apache, SecurityGroup:vnc,
SecurityGroup:appserver2, SecurityGroup:FTP, SecurityGroup:webserver,
SecurityGroup:default, SecurityGroup:test-1228851996]
>>> us_group = groups[0]
>>> us_group
SecurityGroup:alfresco
>>> us_group.rules
[IPPermissions:tcp(22-22), IPPermissions:tcp(80-80), IPPermissions:tcp(1445-1445)]
>>> eu_group = us_group.copy_to_region(eu)
>>> eu_group.rules
[IPPermissions:tcp(22-22), IPPermissions:tcp(80-80), IPPermissions:tcp(1445-1445)]

In the above example, we chose one of the security groups available
in the us-east-1 region (the group alfresco) and copied that security
group to the eu-west-1 region.  All of the rules associated with the
original security group will be copied as well.

If you would like your default region to be something other than
us-east-1, you can override that default in your boto config file
(either ~/.boto for personal settings or /etc/boto.cfg for system-wide
settings).  For example:

[Boto]
ec2_region_name = eu-west-1
ec2_region_endpoint = eu-west-1.ec2.amazonaws.com

The above lines added to either boto config file would set the default
region to be eu-west-1.

Images & Instances
------------------

An Image object represents an Amazon Machine Image (AMI) which is an
encrypted machine image stored in Amazon S3.  It contains all of the
information necessary to boot instances of your software in EC2.

To get a listing of all available Images:

>>> images = conn.get_all_images()
>>> images
[Image:ami-20b65349, Image:ami-22b6534b, Image:ami-23b6534a, Image:ami-25b6534c, Image:ami-26b6534f, Image:ami-2bb65342, Image:ami-78b15411, Image:ami-a4aa4fcd, Image:ami-c3b550aa, Image:ami-e4b6538d, Image:ami-f1b05598]
>>> for image in images:
...    print image.location
ec2-public-images/fedora-core4-base.manifest.xml
ec2-public-images/fedora-core4-mysql.manifest.xml
ec2-public-images/fedora-core4-apache.manifest.xml
ec2-public-images/fedora-core4-apache-mysql.manifest.xml
ec2-public-images/developer-image.manifest.xml
ec2-public-images/getting-started.manifest.xml
marcins_cool_public_images/fedora-core-6.manifest.xml
khaz_fc6_win2003/image.manifest
aes-images/django.manifest
marcins_cool_public_images/ubuntu-6.10.manifest.xml
ckk_public_ec2_images/centos-base-4.4.manifest.xml

The most useful thing you can do with an Image is to actually run it, so let's
run a new instance of the base Fedora image:

>>> image = images[0]
>>> image.location
ec2-public-images/fedora-core4-base.manifest.xml
>>> reservation = image.run()

This will begin the boot process for a new EC2 instance.  The run method
returns a Reservation object which represents a collection of instances
that are all started at the same time.  In this case, we only started one
but you can check the instances attribute of the Reservation object to see
all of the instances associated with this reservation:

>>> reservation.instances
[Instance:i-6761850e]
>>> instance = reservation.instances[0]
>>> instance.state
u'pending'
>>>

So, we have an instance booting up that is still in the pending state.  We
can call the update method on the instance to get a refreshed view of it's
state:

>>> instance.update()
>>> instance.state
u'pending'
>>> # wait a few minutes
>>> instance.update()
>>> instance.state
u'running'

So, now our instance is running.  The time it takes to boot a new instance
varies based on a number of different factors but usually it takes less than
five minutes.

Now the instance is up and running you can find out its DNS name like this:

>>> instance.dns_name
u'ec2-72-44-40-153.z-2.compute-1.amazonaws.com'

This provides the public DNS name for your instance.  Since the 2007--3-22
release of the EC2 service, the default addressing scheme for instances
uses NAT-addresses which means your instance has both a public IP address and a
non-routable private IP address.  You can access each of these addresses
like this:

>>> instance.public_dns_name
u'ec2-72-44-40-153.z-2.compute-1.amazonaws.com'
>>> instance.private_dns_name
u'domU-12-31-35-00-42-33.z-2.compute-1.internal'

Even though your instance has a public DNS name, you won't be able to
access it yet because you need to set up some security rules which are
described later in this tutorial.

Since you are now being charged for that instance we just created, you will
probably want to know how to terminate the instance, as well.  The simplest
way is to use the stop method of the Instance object:

>>> instance.stop()
>>> instance.update()
>>> instance.state
u'shutting-down'
>>> # wait a minute
>>> instance.update()
>>> instance.state
u'terminated'
>>>

When we created our new instance, we didn't pass any args to the run method
so we got all of the default values.  The full set of possible parameters
to the run method are:

min_count - The minimum number of instances to launch.
max_count - The maximum number of instances to launch.
keypair - Keypair to launch instances with (either a KeyPair object or a string with the name of the desired keypair.
security_groups - A list of security groups to associate with the instance.  This can either be a list of SecurityGroup objects or a list of strings with the names of the desired security groups.
user_data - Data to be made available to the launched instances.  This should be base64 encoded according to the EC2 documentation.

So, if I wanted to create two instances of the base image and launch them
with my keypair, called gsg-keypair, I would to this:

>>> reservation.image.run(2,2,'gsg-keypair')
>>> reservation.instances
[Instance:i-5f618536, Instance:i-5e618537]
>>> for i in reservation.instances:
...    print i.status
u'pending'
u'pending'
>>>

Later, when you are finished with the instances you can either stop each
individually or you can call the stop_all method on the Reservation object:

>>> reservation.stop_all()
>>>

If you just want to get a list of all of your running instances, use
the get_all_instances method of the connection object.  Note that the
list returned is actually a list of Reservation objects (which contain
the Instances) and that the list may include recently terminated instances
for a small period of time subsequent to their termination.

>>> instances = conn.get_all_instances()
>>> instances
[Reservation:r-a76085ce, Reservation:r-a66085cf, Reservation:r-8c6085e5]
>>> r = instances[0]
>>> for inst in r.instances:
...    print inst.state
u'terminated'
>>>

A recent addition to the EC2 api's is to allow other EC2 users to launch
your images.  There are a couple of ways of accessing this capability in
boto but I'll show you the simplest way here.  First of all, you need to
know the Amazon ID for the user in question.  The Amazon Id is a twelve
digit number that appears on your Account Activity page at AWS.  It looks
like this:

1234-5678-9012

To use this number in API calls, you need to remove the dashes so in our
example the user ID would be 12345678912.  To allow the user associated
with this ID to launch one of your images, let's assume that the variable
image represents the Image you want to share.  So:

>>> image.get_launch_permissions()
{}
>>>

The get_launch_permissions method returns a dictionary object two possible
entries; user_ids or groups.  In our case we haven't yet given anyone
permission to launch our image so the dictionary is empty.  To add our
EC2 user:

>>> image.set_launch_permissions(['123456789012'])
True
>>> image.get_launch_permissions()
{'user_ids': [u'123456789012']}
>>>

We have now added the desired user to the launch permissions for the Image
so that user will now be able to access and launch our Image.  You can add
multiple users at one time by adding them all to the list you pass in as
a parameter to the method.  To revoke the user's launch permissions:

>>> image.remove_launch_permissions(['123456789012'])
True
>>> image.get_launch_permissions()
{}
>>>

It is possible to pass a list of group names to the set_launch_permissions
method, as well.  The only group available at the moment is the group "all"
which would allow any valid EC2 user to launch your image.

Finally, you can completely reset the launch permissions for an Image with:

>>> image.reset_launch_permissions()
True
>>>

This will remove all users and groups from the launch permission list and
makes the Image private, again.

Security Groups
----------------

Amazon defines a security group as:

"A security group is a named collection of access rules.  These access rules
 specify which ingress, i.e. incoming, network traffic should be delivered
 to your instance."

To get a listing of all currently defined security groups:

>>> rs = conn.get_all_security_groups()
>>> print rs
[SecurityGroup:appserver, SecurityGroup:default, SecurityGroup:vnc, SecurityGroup:webserver]
>>>

Each security group can have an arbitrary number of rules which represent
different network ports which are being enabled.  To find the rules for a
particular security group, use the rules attribute:

>>> sg = rs[1]
>>> sg.name
u'default'
>>> sg.rules
[IPPermissions:tcp(0-65535),
 IPPermissions:udp(0-65535),
 IPPermissions:icmp(-1--1),
 IPPermissions:tcp(22-22),
 IPPermissions:tcp(80-80)]
>>>

In addition to listing the available security groups you can also create
a new security group.  I'll follow through the "Three Tier Web Service"
example included in the EC2 Developer's Guide for an example of how to
create security groups and add rules to them.

First, let's create a group for our Apache web servers that allows HTTP
access to the world:

>>> web = conn.create_security_group('apache', 'Our Apache Group')
>>> web
SecurityGroup:apache
>>> web.authorize('tcp', 80, 80, '0.0.0.0/0')
True
>>>

The first argument is the ip protocol which can be one of; tcp, udp or icmp.
The second argument is the FromPort or the beginning port in the range, the
third argument is the ToPort or the ending port in the range and the last
argument is the CIDR IP range to authorize access to.

Next we create another group for the app servers:

>>> app = conn.create_security_group('appserver', 'The application tier')
>>>

We then want to grant access between the web server group and the app
server group.  So, rather than specifying an IP address as we did in the
last example, this time we will specify another SecurityGroup object.

>>> app.authorize(src_group=web)
True
>>>

Now, to verify that the web group now has access to the app servers, we want to
temporarily allow SSH access to the web servers from our computer.  Let's
say that our IP address is 192.168.1.130 as it is in the EC2 Developer
Guide.  To enable that access:

>>> web.authorize(ip_protocol='tcp', from_port=22, to_port=22, cidr_ip='192.168.1.130/32')
True
>>>

Now that this access is authorized, we could ssh into an instance running in
the web group and then try to telnet to specific ports on servers in the
appserver group, as shown in the EC2 Developer's Guide.  When this testing is
complete, we would want to revoke SSH access to the web server group, like this:

>>> web.rules
[IPPermissions:tcp(80-80),
 IPPermissions:tcp(22-22)]
>>> web.revoke('tcp', 22, 22, cidr_ip='192.168.1.130/32')
True
>>> web.rules
[IPPermissions:tcp(80-80)]
>>>