Quick N Dirty way to Map Commands to remote servers via ssh

You may be running several independent but similar servers at the same time and wasting time by executing commands in all of them one by one.

Wouldn’t it be nice to send a command to all of them at once? or to monitor all of them at once.

The following script can be used as a building block to more complex automation tasks for a small size set of servers. (If you’re managing over 50 servers, I’d probably consider looking a different way to arrange servers (map/reduce cluster), but if you’re doing something below that number this might suffice)

[code lang=”python”]
#!/usr/bin/python

#########################################################
# Author: Angel Leon (gubatron@gmail.com) – October 2009
#
# Invokes a command locally and invokes the same command
# in all machines under the specified username, servers
#
# Requirement: Have a public ssh_key for that user on all
# the other machines so you don’t have to authenticate
# on all the other machines.
#########################################################
import sys
import os

# set the username that has access to all the machines here
user=’safeuser’

# add all your server names here
servers=[‘server1.mydomain.com’,’server2.mydomain.com’,’server3.mydomain.com’]

if __name__ == "__main__":
if len(sys.argv) < 2:
print "Usage: ssh_map_command <cmd>"
sys.exit(0)

cmd= ‘ ‘.join(sys.argv[1:])

#Execute locally first
print cmd
os.system(cmd)

#Execute for all the servers in the list
for server in servers:
remote_cmd="ssh %s@%s %s" % (user,server,cmd)
print remote_cmd
os.system(remote_cmd)
print
[/code]

Save as ssh_map_command and chmod +x it.

Sample uses
Check the average load of all machines at once (then use output to mitigate high load issues)
[code lang=”shell”]$ ssh_map_command uptime[/code]

Send HUP signal to all your web servers (put it in an alias or other script… and that’s how you start building more complex scripts)
[code lang=”shell”]$ ssh_map_command ps aux | grep [l]ighttpd | kill -HUP `awk {‘print $2’}`[/code]

Check if processes are alive, check memory usage on processes across different machines, grep remote all logs at once, svn up on all machines, rsync from one to many, hey, you can even tail -f and grep all the logs at once, you can go nuts with this thing. Depends on what you need to do.

Requirements

Security Advisory
Make sure only the desired user has read/write/execute access to it and keep your private ssh keys safe (preferably only read and execute for the owner, and no permissions whatsoever to anybody else chmod 500 ssh_mod_map), if possible change them as often as possible, for it may become a big security whole if an attacker can manage to write code on this script, specially if you have cronjobs invoking it. Your attacker would only need to change code here to mess up all of your machines.

Disclaimer and Call for Knowledge
Please, if someone knows of a standard way to map commands to multiple servers, please let me know in the comment section, in my case I needed a solution and I wrote a quick and dirty python script and tried to secure it as best as I could, by no means I’m saying that this is the best solution to mapping commands, in fact I believe it might be the least efficient way, however it works good enough for my personal needs.

Ubuntu/Debian Quick Reference: How To Change Your Server’s UTC Timezone on the command line

Just Type…
sudo dpkg-reconfigure tzdata

…and follow the instructions on screen.

The process should look something like the following:


Select your Region


Select a city on your time zone


You’re done.

Tip
You can always check the status of your configuration using
sudo debconf-show tzdata

You could for example map that command via ssh to several machines and grep for “*”, that way you could easily spot servers with wrong timezones very quickly.

Compiling Ghost White Crab on Linux x86_64 Ubuntu Linux

Compiling this software is usually just a matter of running ‘make’, but if you try to do so in Ubuntu 64bit, you’ll encounter the following error on the config_test.log file

skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.3.3/libgcc.a when searching for -lgcc

To solve this issue just do the following:

apt-get install libc6-dev-i386

This makes the make script work since lib6-dev-i386 installs the
l32bit development libraries for GNU C for AMD64 (64bit linux)

Now it works!

About Ghost White Crab
GhostWhiteCrab is a portable, stand-alone GWebCache implemented in C. It is open-source and distributed under a BSD-style license. It was developed as a light-weight replacement for CGI GWebCache implementations in PHP, Perl, ASP etc. Unlike those, GhostWhiteCrab can easily handle millions of transactions per hour while consuming little memory and CPU resources. Although GWebCaches are usually used as bootstrap system for Gnutella, you can use them for any other network as well. GhostWhiteCrab doesn’t handle multiple networks at a time, but you can configure it for any network you like.

GhostWhiteCrab is also an UDP Hostcache (UHC) for the Gnutella network. This is the recommended use now as most Gnutella clients support this or have completely switched to this bootstrap method. An UDP hostcache is much lighter weight, especially in terms of traffic. It’s possible to run GhostWhiteCrab as UHC instead of a GWebCache, or it can serve as both at the same time.

Download BZIP2 compressed sources

Geek T-Shirt Collection #1 – The Ubuntu Upgrade T-Shirt

This is the first of a series of picture posts I intend to share with you so you can take a look at the collection of Geek T-Shirts I’ve managed to amass during the last 5 years.

If you are an internet or technology company, feel welcome to send me a t-shirt, If I haven’t done a review of your company/product, I’ll do a review for you in return and I’ll take a picture of myself wearing it.

The Ubuntu Upgrade T-Shirt


Front – “sudo apt-get upgrade”


Back – Ubuntu logo

This T-shirt I made as an homage to Ubuntu and the mantra of easy updates, as well as a mantra of always keeping yourself up to date in life with whatever you do.

Nowadays updating Ubuntu is even easier, but some people like me still might do it old style, updating manually the /etc/apt/sources.list file, and running sudo apt-get update and then sudo apt-get upgrade dist, the t-shirt reads sudo apt-get upgrade which will basically upgrade those packages on your machine that need to do so.

The T-shirt was made at my favorite T-shirt-making website, (here goes the free ad) SpreadShirt.com

Photographed by Paulina Leon at 5 Points in Long Island City, NY.

See the Next T-Shirt

Checking the Speed of your network interface

I recently requested an upgrade on one of our dedicated server’s uplink speed, we only had a 10Mbps Uplink, we requested an upgrade to 100Mbps to serve a lot more.

How do you verify the upgrade has been done correctly?

As root, issue the following comand:

# mii-tool
eth0: negotiated 10baseT-FD, link ok

If it doesn’t work (for debian or ubuntu), make sure you have installed the net-tools package (The NET-3 networking toolkit)

Blooploader 0.6 is Hardy compatible


Blooploader 0.6 running on Hardy. Currently available only via subversion.

For our Linux users, you can safely update to Ubuntu Hardy if the one thing holding your breath was compatibility with the Blooploader.

Currently we run the Blooploader in Linux from source, you just need to have installed, Qt4, sip4, and PyQt4 on your machine. If you are an Ubuntu user this translates to:

  1. Checking out the source from our subversion repository
  2. sudo apt-get install python-sip4 python-qt4 python-qt-4-common
  3. ./run

For those of you that want to try the Blooploader in Ubuntu, and you have no clue on how to use the command line, we promise we’ll have a new .deb installer for our next release now that Hardy has enabled binary packages on their repository for all our dependencies.