Here’s a script I had to write at work for moving files into a backup directory if they’d been moved to S3. It was very interesting for me to write as it used the holy grail of bash “pipe, sed, && awk”!

#!/bin/bash
site=$1
file=$2;
# Establish the name of the file.
name=`basename "$file"`;
# Establish the numbered directory to backup into.
dir=`dirname "$file" | awk '{print $NF}' FS=\/`
# Form the backup location.
backup="/data/images/$site/backup/"$dir"/"
# Create the backup location (if it doesnt exist).
mkdir -p $backup;
# Check the file exists on S3
uri=`echo http://images.$site.com.au.s3.amazonaws.com$file | sed "s/ /%20/g"`

curl -I $uri 2>&1 | grep 'HTTP/1.1 200'
if [ $? = 0 ]; then
# More user feedback
echo "Moving "$file" into "$backup$name;
# Perform the move (@todo logging & deletion based on $mode).
mv "$file" "$backup$name";
else
echo "Warning: file doesnt exist on S3 ($uri)";
fi
echo ""; # A new line just to space the feedback out a bit.