You are here: Home Blog Topics boto

boto

May 02, 2013

Automated Amazon EBS Volume Snapshots with Boto

by Chris Crownhart — last modified May 02, 2013 09:40 AM
Filed Under:

The following write-up describes how to setup, configure and use an easy tool for taking regular snapshots of Amazon Elastic Block Storage (EBS) volumes.

Recently, we were asked by a client to setup regular EBS volume snapshots for their production EC2 instances. After a little research, Mike came across a fairly simple tool called aws-snapshot-tool: https://github.com/evannuil/aws-snapshot-tool.  This tool runs as a cron job and automatically creates daily, weekly, and monthly snapshots, and automatically rotates out the oldest snapshots for each time period.

The tool, written in Python, takes advantage of an excellent module called boto (https://github.com/boto/boto, http://docs.pythonboto.org/en/latest/).  Boto is a Python package that provides an interface to Amazon Web Services (AWS) allowing you to easily maintain and manage various aspects of your AWS environment with very little coding.

Install Notes

The aws-snapshot-tool github page has simple instructions to get started, but here are a couple points of interest:

install and setup boto

  1. pip install boto
  2. Setup a .boto config file to store your AWS access key (See: https://code.google.com/p/boto/wiki/BotoConfig)

Setup your Identity and Access Management (IAM) access

When you access AWS through their API, you are utilizing the web service functions to which your account has been given access.  The aws-snapshot-tool includes a sample IAM configuration, that shows which actions need to be setup:

"Action": [
        "ec2:CreateSnapshot",
        "ec2:CreateTags",
        "ec2:DeleteSnapshot",
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeSnapshots",
        "ec2:DescribeTags",
        "ec2:DescribeVolumeAttribute",
        "ec2:DescribeVolumeStatus",
        "ec2:DescribeVolumes"
      ],

Configuration (including rotation)

Configuration is very easy.  The script uses a config.py file that looks like:

config = {
    'ec2_region_name': 'us-east-1',
    'ec2_region_endpoint': 'ec2.us-east-1.amazonaws.com',
    'tag_name': 'tag:MakeSnapshot',
    'tag_value':    'True',
    'keep_day': 5,
    'keep_week': 5,
    'keep_month': 11,
    'log_file': '/tmp/makesnapshots.log',
}

There are three important pieces being defined here:

  • The region where your EBS volumes are defined
  • The tag name put on each volume in order to enable snapshots
  • The definition for how long to keep snapshots: 5 days, 5 weeks, and 13 months

Tag Your Volumes

Well, it couldn't be any easier.  Simply add a Tag of 'MakeSnapshot', with a value of 'True', (as defined in the above config file) to any volume you want picked up by this script.  The script loops through all volumes in the entire defined ec2_region_name field, looks for that tag, and handles the snapshot and rotation of snapshots automatically.

AWS Volume Snapshot Tag

Cron Setup & Output

We chose to run our script on another EC2 instance, but this could run anywhere.   We setup the following cron jobs to run:

$ crontab -l

# mon-fri 30 3 * * 1-5 /home/zope/aws-snapshot-tool/makesnapshots.py day

# every sat 30 3 * * 6 /home/zope/aws-snapshot-tool/makesnapshots.py week

# first sun 30 3 1-7 * 0 /home/zope/aws-snapshot-tool/makesnapshots.py month

Each snapshot is created with a description that begins like 'day_snapshot...', 'week_snapshot...', or 'month_snapshot...', like in the following screenshots:

AWS Snapshots - Day

AWS Snapshots - Week

Gotchas?

At the moment, I can only think of 3:

  • The config.py script points to a region. If we decide to start using other regions for our servers, we will need to accomodate that.
  • The cost of storage. Is cheap. However, once this has run for a year, we will have 23 snapshots per volume saved in our account.
  • These are real time snapshots. It is each server administrator's responsibility to insure the volume being backed up is in the proper state.

Apr 08, 2013

How to open SSH links in iTerm 2

by Mike Cullerton — last modified Apr 08, 2013 04:45 PM
Filed Under:

When I switched to iTerm 2, it wouldn't open ssh links for me. Instead, it opened new tabs. I finally solved the problem with a simple configuration change.

I know some of you have this problem. I've seen a few references to it on the net, but no solutions. iTerm 2 won't open ssh links.

I have a pyramid application that uses boto to query ec2 instances and then posts information about the instances--including an ssh link, to a web page on my laptop.

The ssh links open fine in iTerm. I couldn't get them to open in iTerm 2 though. Instead, they just opened new tabs.

I really like using iTerm 2, so to 'solve' the problem, I added the actual ssh shell command for each of the instances as text on the page.

To open an ssh connection to an instance, I simply copy the ssh command, click the link, paste the ssh command into the new terminal tab, and hit return.

Hey, it works.

Every once in a while, when it really bothers me, I try to figure out a real solution.

Today was one of those days.

The solution turned out to be pretty simple.

In the iTerm 2 Preferences, under the Profiles tab, General section, change the Command to Login shell.

Now, when I click on one of the links, it opens correctly in iTerm 2.

I hope it works for you too.

*Note that you must first have iTerm 2 set up to accept ssh connections. Preferences->Profiles->General->URL Schemes

 

 
 
Copyright © 2003-2017 Core Software Group | 303/809-1001 | Fort Collins, Colorado | All rights reserved.