McDermottroe.com

Archive for the "Tech" Category

SFTP to EC2 with Python and Boto

Monday, November 14th, 2011

Today I wanted to automate the upload of some code to a new Amazon EC2 instance. I’ve been scripting the rest of the job using Boto but when I was lazily looking for an example of how to do SFTP with Boto there wasn’t anything obvious in the first few pages of Google’s results. So, here’s the snippet for any other lazy coders out there:

import boto.manage.cmdshell
 
def upload_file(instance, key, username, local_filepath, remote_filepath):
    """
    Upload a file to a remote directory using SFTP. All parameters except for
    "instance" are strings. The instance parameter should be a
    boto.ec2.instance.Instance object.
 
    instance        An EC2 instance to upload the files to.
    key             The file path for a valid SSH key which can be used to log
                    in to the EC2 machine.
    username        The username to log in as.
    local_filepath  The path to the file to upload.
    remote_filepath The path where the file should be uploaded to.
    """
    ssh_client = boto.manage.cmdshell.sshclient_from_instance(
        instance,
        key,
        user_name=username
    )
    ssh_client.put_file(local_filepath, remote_filepath)

Boto depends on paramiko to handle the SSH parts, so you’ll need that installed too.

Profiling PHP with XHProf

Tuesday, July 6th, 2010

If you find yourself writing any performance sensitive code in PHP, you probably want a profiler to tell you where the slowest parts of your code are. Sometimes you can get by with educated guessing and a few well-placed uses of echo and time, but there really is no substitute for hard data. Luckily, Facebook have written and released a profiler for PHP and it’s pretty easy to use.

First off, download and install it. It’s a PECL extension to PHP, so it should install like any other PECL extension you have. I’m developing on top of FreeBSD, so I made a port for it. It’s not yet in the ports tree, but if you’re running PHP on FreeBSD, you can extract the port out of the PR I filed. It’s in the ports tree as devel/pecl-xhprof.

Once you have it installed, here’s how you use it:

// Start the profiler
//
// XHPROF_FLAGS_MEMORY adds memory usage data, it's quite useful.
// See the docs for further flags.
xhprof_enable(XHPROF_FLAGS_MEMORY);
 
// Put the bulk of your code here
 
// Stop the profiler and get the profile data
$profile_data = xhprof_disable();

After you get the profile data you can either save it somewhere and use the XHProf UI provided to browse the data or you can just process the data directly. I’m working on vBulletin, so I integrated it into the vBulletin debug output. If you’re doing the processing yourself, the following snippet is useful for converting the inclusive times returned by xhprof_disable() to exclusive times.

require_once('/path/to/xhprof/display/xhprof.php');
$profile_data_totals = array(); // Will contain data for the whole script
$profile_data_exclusive = xhprof_compute_flat_info($profile_data, $profile_data_totals);

That’s pretty much it. For anything more than that, refer to the XHProf documentation or have a dig through the XHProf and XHProf UI sources.

Snippets

Thursday, June 24th, 2010