startKeychain – bash utility to start ssh-agent

For my (and your) future reference, here’s a function to put on your .bashrc or .bash_profile, you can invoke it later at any time to start/re-start your ssh-agent.


[bash]
function startKeychain {
killall ssh-agent
rm ~/.keychain/*
keychain id_rsa
HOSTNAME=`hostname`
source ~/.keychain/${HOSTNAME}-sh
}
[/bash]

Then at any time, the “command” startKeychain will be available on your command line.

Output should look something like this:
[bash]

gubatron@gubatron-desktop:~$ startKeychain

KeyChain 2.6.8; http://www.gentoo.org/proj/en/keychain/
Copyright 2002-2004 Gentoo Foundation; Distributed under the GPL

* Initializing /home/gubatron/.keychain/gubatron-desktop-sh file…
* Initializing /home/gubatron/.keychain/gubatron-desktop-csh file…
* Initializing /home/gubatron/.keychain/gubatron-desktop-fish file…
* Starting ssh-agent
* Initializing /home/gubatron/.keychain/gubatron-desktop-sh-gpg file…
* Initializing /home/gubatron/.keychain/gubatron-desktop-csh-gpg file…
* Initializing /home/gubatron/.keychain/gubatron-desktop-fish-gpg file…
* Starting gpg-agent
* Adding 1 ssh key(s)…
Identity added: /home/gubatron/.ssh/id_rsa (/home/gubatron/.ssh/id_rsa)

[/bash]

Comments are welcome to improve it, I’m not an ssh-agent expert, but this seems to do the work.

[SOLVED] Issue with KDE 4.4.2 and Dolphin always asking my ssh passwords whenever I browsed folder I checked out from a remote subversion repository.

Quick N Dirty way to Map Commands to remote servers via ssh

You may be running several independent but similar servers at the same time and wasting time by executing commands in all of them one by one.

Wouldn’t it be nice to send a command to all of them at once? or to monitor all of them at once.

The following script can be used as a building block to more complex automation tasks for a small size set of servers. (If you’re managing over 50 servers, I’d probably consider looking a different way to arrange servers (map/reduce cluster), but if you’re doing something below that number this might suffice)

[code lang=”python”]
#!/usr/bin/python

#########################################################
# Author: Angel Leon (gubatron@gmail.com) – October 2009
#
# Invokes a command locally and invokes the same command
# in all machines under the specified username, servers
#
# Requirement: Have a public ssh_key for that user on all
# the other machines so you don’t have to authenticate
# on all the other machines.
#########################################################
import sys
import os

# set the username that has access to all the machines here
user=’safeuser’

# add all your server names here
servers=[‘server1.mydomain.com’,’server2.mydomain.com’,’server3.mydomain.com’]

if __name__ == "__main__":
if len(sys.argv) < 2:
print "Usage: ssh_map_command <cmd>"
sys.exit(0)

cmd= ‘ ‘.join(sys.argv[1:])

#Execute locally first
print cmd
os.system(cmd)

#Execute for all the servers in the list
for server in servers:
remote_cmd="ssh %s@%s %s" % (user,server,cmd)
print remote_cmd
os.system(remote_cmd)
print
[/code]

Save as ssh_map_command and chmod +x it.

Sample uses
Check the average load of all machines at once (then use output to mitigate high load issues)
[code lang=”shell”]$ ssh_map_command uptime[/code]

Send HUP signal to all your web servers (put it in an alias or other script… and that’s how you start building more complex scripts)
[code lang=”shell”]$ ssh_map_command ps aux | grep [l]ighttpd | kill -HUP `awk {‘print $2’}`[/code]

Check if processes are alive, check memory usage on processes across different machines, grep remote all logs at once, svn up on all machines, rsync from one to many, hey, you can even tail -f and grep all the logs at once, you can go nuts with this thing. Depends on what you need to do.

Requirements

Security Advisory
Make sure only the desired user has read/write/execute access to it and keep your private ssh keys safe (preferably only read and execute for the owner, and no permissions whatsoever to anybody else chmod 500 ssh_mod_map), if possible change them as often as possible, for it may become a big security whole if an attacker can manage to write code on this script, specially if you have cronjobs invoking it. Your attacker would only need to change code here to mess up all of your machines.

Disclaimer and Call for Knowledge
Please, if someone knows of a standard way to map commands to multiple servers, please let me know in the comment section, in my case I needed a solution and I wrote a quick and dirty python script and tried to secure it as best as I could, by no means I’m saying that this is the best solution to mapping commands, in fact I believe it might be the least efficient way, however it works good enough for my personal needs.

Python Script to Update WordPress in One Step

During the past week, I think I had to update all my wordpress instances twice, and it’s become really annoying doing this manually. I’ve written a python script which I’ll share with you.

How I keep my wordpress updated by hand
I tend to keep my wp-content folder outside of my wordpress installation for 2 reasons:

1. I don’t like to loose my themes, plugins and customizations
2. I like to keep all my customization changes under subversion

So, if I had my wordpress installation say at:
/home/user/public_html/blog

I’d keep my wp-content folder for that here:

/home/user/public_html/wp-content-for-blog

So when I upgrade my blog, I always remove the original wp-content folder that comes along wordpress, and I symlink my hard worked on wp-content folder that lives outside to the freshly unzipped wordpress folder.

[bash]
user@machine:~/public_html/blog$ ls -l

lrwxrwxr-x 1 user www 54 2008-11-26 09:29 wp-content -> /home/user/public_html/wp-content-for-blog

[/bash]

So what I endup doing all the time, is downloading the latest.zip to ~/public_html/, it will unzip under ~/public_html/wordpress, and then I’ll copy the current ~/public_html/blog/wp-config.php to ~/public_html/wordpress, then I’ll remove the default ~/public_html/wordpress/wp-content and symlink the outer wp-content with all my customizations, themes and plugins to it. Once done, I’ll make a backup of the old wordpress folder, and then I’ll rename wordpress folder to the name of the blog folder, and it’s all done.

It’s simple, but when you have to do it for 5 blogs, every week, it’s not fun anymore.

The Update Script

So here’s a script to do it in one step. If you’re not using my symlinked technique, this will do it for you, you only need to specify the full path to the folder where you want to keep your current wp-content folder outside the new installation before you apply the update, and the name of the folder where your current blog lives. The script below will have its configuration variables towards the beginning set so that they are in line with the example I’ve been talking about.

[python]
#!/usr/bin/python
#########################################################################################
#
# upgrade_wordpress.py – Script to automatically upgrade your wordpress installation.
#
# Requirements:
# – Python 2.4 or older
# – WordPress should already be installed
# – CURL (sudo apt-get install curl)
#
# Author: Angel (Gubatron) Leon
# LICENSE: See the GPL2 license.
# 2008
#########################################################################################
import os

#########################################################################################
#Config (relative to the folder where this script will be run from)
#########################################################################################

#The current folder where the blog lives
BLOG_FOLDER=’blog’

#
# The first time you run the script, it will try to make a copy of your
# current wp-content folder outside. Copy here the location of where
# the wp-content folder with your themes and plugins should exist.
#
# After it unzips, it will remove the default wp-content folder from
# the new installation, and it will symlink the external wp-content
# That way you don’t ever have to worry about loosing your customizations
# and plugins.
#
WP_CONTENT_OUTSIDE_COPY_FOLDER="/home/user/public_html/wp-content-for-blog"

#This is where a backup of your current blog will be
BLOG_FOLDER_BACKUP_FOLDER=BLOG_FOLDER+’.old’

#Where to download the wordpress latest.zip from
WORDPRESS_LATEST_ZIP_URL=’http://wordpress.org/latest.zip’

#### DO NOT MODIFY AFTER THESE LINES ####

def downloadWordpress(url=WORDPRESS_LATEST_ZIP_URL):
if os.path.exists(‘latest.zip’):
print "Removing old latest.zip"
os.remove(‘latest.zip’)

#Try to download with CURL
print "Attempting to download latest.zip from wordpress.org"
os.system(‘curl %s -o latest.zip’ % url)

if not os.path.exists(‘latest.zip’):
os.system(‘wget ‘ + url)

return os.path.exists(‘latest.zip’)

def dirExists(dirName):
return os.path.exists(dirName) and os.path.isdir(dirName)

def backupBlog(currentBlogFolder=BLOG_FOLDER,
wpContentOriginalFolder=WP_CONTENT_OUTSIDE_COPY_FOLDER,
backupFolder=BLOG_FOLDER_BACKUP_FOLDER):

#Remove any previous backups
if os.path.exists(backupFolder) and os.path.isdir(backupFolder):
print "Removing previous backup folder"
os.system(‘rm -fr ‘ + backupFolder)

#Copy the current blog folder into a backup folder just in case.
#We won’t do any database backups for now.
print "Creating new backup folder"
os.system(‘cp -r %s %s’ % (currentBlogFolder,backupFolder))

#Check for the copy of wp-content outside the blog, if it doesn’t exist
#we’ll make it for the first time.
if not dirExists(wpContentOriginalFolder):
print "Creating outside copy of wp-content"
os.system(‘cp -r %s %s’ % (os.path.join(currentBlogFolder,’wp-content’),
wpContentOriginalFolder))

#Copy the latest wp-config.php outside to the current folder
print "Copying your latest wp-config.php outside"
os.system(‘cp %s .’ % (os.path.join(currentBlogFolder,’wp-config.php’)))

backupFolderExists = dirExists(backupFolder)
wpContentFolderExists = dirExists(wpContentOriginalFolder)
configFileExists = os.path.exists(‘wp-config.php’)

return backupFolderExists and wpContentOriginalFolder and configFileExists

def upgradeBlog(currentBlogFolder=BLOG_FOLDER,
backupFolder=BLOG_FOLDER_BACKUP_FOLDER,
url=WORDPRESS_LATEST_ZIP_URL,
wpContentOriginalFolder=WP_CONTENT_OUTSIDE_COPY_FOLDER):

if not downloadWordpress(url):
print "Could not download latest.zip, aborting."
return False

if not backupBlog(currentBlogFolder,wpContentOriginalFolder,backupFolder):
print "Could not backup blog or wp-config.ph, aborting."
return False

if currentBlogFolder == ‘wordpress’:
print "The current blog folder cannot be ‘wordpress, aborting."
return False

#1. If a wordpress/ folder exists, wipe it.
if dirExists(‘wordpress’):
print "Removing old wordpress folder"
os.system(‘rm -fr wordpress’)

if dirExists(‘%s.delete’ % currentBlogFolder):
print "Removing old %s.delete folder" % currentBlogFolder
os.system(‘rm -fr %s.delete folder’ % currentBlogFolder)

#2. Unzip new copy
os.system(‘unzip latest.zip’)

if not dirExists(‘wordpress’):
print "Could not unzip the wordpress installation, aborting."
return False

#1. Copy wp-config.php into the new installation
os.system(‘cp wp-config.php wordpress/’)

#2. Remove the default wp-content folder
os.system(‘rm -fr wordpress/wp-content’)

#3. Symlink the original wp-content that lives outside
os.system(‘ln -s %s wordpress/wp-content’ % (wpContentOriginalFolder))

#4. Verify symlink was created
if not (os.path.exists(‘wordpress/wp-content’) and os.path.islink(‘wordpress/wp-content’)):
print "Could not create symlink to wp-content, aborting."
return False

#5. Move original folder to folder.delete, and make this wordpress folder the current folder.
os.system(‘mv %s %s.delete’ % (currentBlogFolder,currentBlogFolder))

if not dirExists(currentBlogFolder + ".delete"):
print "Could not rename current folder for later deletion, aborting."
return False

#6. Rename the new installation as the current blog
os.system(‘mv %s %s’ % (‘wordpress’,currentBlogFolder))

if dirExists(‘wordpress’):
print "ALERT: The wordpress folder still exists."
return False

if not dirExists(currentBlogFolder):
print "ALERT: The blog doesn’t exist, recover from the backup folder %s please" % (backupFolder)
return False

#7 Cleanup
os.system(‘rm -fr %s.delete’ % (currentBlogFolder))

return True

if __name__ == ‘__main__’:
upgradeBlog()
[/python]

Requirements

  • shell access to the machine where you have your wordpress installed
  • a python interpreter installed
  • curl (sudo apt-get install curl) to download the zip. If you don’t have it it’ll attempt to use wget
  • Installation

  • Right outside your wordpress installation folder, create a new file called upgrade_wordpress.py
  • Copy and paste the script inside that file
  • Edit the configuration variables to point to the name of your wordpress installation folder, and give it a full path to where you want to keep your wp-content folder (including the name of the folder, so if you want to name it the same way, you could do for example /home/user/wp-content and it’ll be saved right under your home)
  • Usage:
    [bash]python upgrade_wordpress.py[/bash]

    The script is very fault proof, it will always try to abort in case something is not going the way it’s expected. At the end of the day it’ll also leave a backup copy of your current blog in case something goes bad, you can always recover.