How to resize an EBS (xfs formatted) partition

First of all, create a snapshot of your EBS volume. Then out of that snapshot you will be able to create your new volume.

However, when you detach the old one from your instance and attach the new one, you will still see the old available space with df

look at my /dev/xvdf/ available space (after mounting the new EBS volume)

ubuntu@ip-10-47-167-74:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       16G  5.9G  9.1G  40% /
udev            7.4G   12K  7.4G   1% /dev
tmpfs           1.5G  176K  1.5G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.4G     0  7.4G   0% /run/shm
/dev/xvdf        20G   19G  1.7G  92% /media/ebs/data  <<< this one

still 20G, I mounted a 80G one!

This drive has been formatted to use an xfs file system. In order to resize it, this is the command I used:

sudo xfs_growfs -d /media/ebs/data

output should be something like this

ubuntu@ip-10-47-167-74:~$ sudo xfs_growfs -d /media/ebs/data
meta-data=/dev/xvdf              isize=256    agcount=4, agsize=1310720 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=5242880, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
data blocks changed from 5242880 to 20971520

now let’s see the df -h output

ubuntu@ip-10-47-167-74:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       16G  5.9G  9.1G  40% /
udev            7.4G   12K  7.4G   1% /dev
tmpfs           1.5G  176K  1.5G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.4G     0  7.4G   0% /run/shm
/dev/xvdf        80G   19G   62G  23% /media/ebs/data

AWS troubleshooting: how to fix a broken EBS volume (bad superblock on xfs)

As great as EBS volumes are on Amazon Web Services, they can break and not ever mount again, even though your data could still be there intact, a simple corruption on the filesystem structure can cause a lot of damage. On this post I teach you how to move all that data onto a new EBS drive, so keep calm and read slowly.

So, you try to mount your drive after some updates and you get an error like this on dmesg | tail:

[56439860.329754] XFS (xvdf): Corruption detected. Unmount and run xfs_repair

so you unmount your drive, invoke xfs_repair and you get this…

$ sudo xfs_repair -n /dev/xvdf
Phase 1 - find and verify superblock...
bad primary superblock - bad magic number !!!

attempting to find secondary superblock...
..........................................

and no good secondary superblock is found.

Don’t panic, this is what you have to do next to solve this issue:

  1. Go to your AWS dashboard, EC2 section.
  2. Click on “Volumes”
  3. Find the broken volume.
  4. Create a snapshot of the broken volume (this takes a while)
  5. Create a new volume the same size (or larger than) your old drive out of the snapshot you just created (this takes a while)
  6. Attach your new volume to the same EC2 instance (no need to reboot or anything), if the old drive was mapped to /dev/xvdf, the new one will be mapped to /dev/xvdg (see how the last letter increases alphabetically)

Now here’s a gotcha, Amazon will not create your new drive using the same file system type (xfs), for some reason it will create it using the ext2 filesystem.

$ sudo file -s /dev/xvdg
/dev/xvdg: Linux rev 1.0 ext2 filesystem data (mounted or unclean), UUID=2e35874f-1d21-4d2d-b42b-ae27966e0aab (large files)

Here you have two options:
1. Live with the new ext2 file system, make sure your /etc/fstab is updated to look something like this:
/dev/xvdg /path/to/mount/to auto defaults,nobootwait,noatime 0 0

or 2. copy the contents of your drive to a temporary location, usually inside /mnt which has plenty of space from that ephemeral drive the ec2 instances come to, and then mkfs.xfs the new volume, and then copy the contents back… (which what I did, as I chose to create a larger drive and the ext2 format that came on the new volume only recognized the size of the snapshot)

Hope this saved your ass, leave a note if I did.

Remember to never do any irreversible action until you have a disk snapshot, try your best to never lose data.

How to have a Play framework app autostart during boot on Elastic Beanstalk CentOS ec2 instances

So you’ve created an Elastic Beanstalk environment, you have a play framework distribution which you’ve created using play dist (either on your local environment, or right there on the server, whatever you prefer)

play dist outputs a my-app-1.0.zip file which has a self-contained version of your app with all the necessary libraries and a start script.

Afer you unzip it, you end up with a my-app-1.0/lib/ folder and a start script.

[ec2-user@ip-10-235-8-106 bullq-1.0]$ ls -l
total 24
drwxrwxr-x 2 ec2-user ec2-user 4096 Sep 27 15:35 lib
-rwxrwxr-x 1 ec2-user ec2-user 4328 Sep 27 15:35 start

Make sure it’s executable by using chmod +x start on the start script.

So now, this is all in the first ec2 instance of your elastic beanstalk environment, if you’re like me and you’ve used ubuntu/debian for your server management things can be slightly different here, since Amazon preferred CentOS for their default image, and here I’ll show you how to make your play app auto start when the server boots because you want every new machine that may be instanciated to have your app installed and to start the service as soon as the machine is up.

Create a /etc/init.d/myappd script
(I’m using ‘myapp’ here as an example, your app can be named whatever is named, so replace accordingly)

#!/usr/bin/env bash
#myappd
#Script to start|stop|restart myappd from /etc/init.d/
#By Gubatron – @gubatron – gubatron@gmail.com

#replace accordingly in these variables ‘myapp’ for the name of your app
PID_FILE=/home/ec2-user/myapp/dist/myapp-1.0/RUNNING_PID
DAEMON_NAME=myappd
DAEMON_PATH=/home/ec2-user/myapp
DAEMON=$DAEMON_PATH/dist/myapp-1.0/start

test -x $DAEMON || exit 0

set -e

function killDAEMON() {
echo “start kill daemon”
kill -9 cat /home/ec2-user/bullq/dist/bullq-1.0/RUNNING_PID
echo “end kill daemon”
}

function removePIDFile() {
if [ -e $PID_FILE ]
then
rm -f $PID_FILE
fi
}

case $1 in
start)
removePIDFile
echo “Starting $DAEMON_NAME… $DAEMON”
nohup $DAEMON &
;;
restart)
echo “Hot restart of $DAEMON_NAME”
killDAEMON
removePIDFile
COMMAND=”nohup $DAEMON &”;
echo $COMMAND
$COMMAND
rm -f $PID_FILE
;;
stop)
echo “Stopping $DAEMON_NAME”
killDAEMON
removePIDFile
;;
*)
echo “Usage: $DAEMON_NAME {start|restart|stop}” >&2
exit 1
;;
esac

exit 0

 

Wire it to autostart

The simplest way I found to have this script start when the server would boot was to add it at the end of the
/etc/rc.local file. (In ubuntu you’d register the new script with the upate-rc.d command)

#!/bin/sh
#
This script will be executed after all the other init scripts.
You can put your own initialization stuff in here if you don’t
want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

/etc/init.d/myappd start

 

can’t ssh to ec2 ubuntu instance, /etc/fstab breaks bootup due to missing ebs volume [SOLVED]

Screen Shot 2013-08-21 at 12.08.04 PM

So the /etc/fstab file on your root volume looked like this

LABEL=cloudimg-rootfs / ext4 defaults 0 0
/dev/xvdf /mnt/backups auto defaults,comment=cloudconfig 0 2

by mistake you deleted the ebs volume that you had mounted on /mnt/backups (or whatever folder) and you restarted your ubuntu instance not knowing that if the /etc/fstab would break it would not continue to start all the application layer networking services like ssh on port 22…

you can ping the machine, but you can’t ssh, amazon support won’t respond or will tell you to fuck yourself.

you learn that ubuntu has had this bug for a while, but it’s been addressed by passing your volume configuration a nobootwait option.

you wish your /etc/fstab looked like this, but you can’t get in, amazon doesn’t give you any other options from their console to go in and solve the problem through a console…

LABEL=cloudimg-rootfs / ext4 defaults 0 0
/dev/xvdf /mnt/backups auto defaults,nobootwait,comment=cloudconfig 0 2

No worries, I have a fix that will let you edit that file, and boot back and try to recover things, you may have lost that ebs volume, but you won’t have to setup this computer again.

1. Make a snapshot of the root volume on that instance. This will take a while.
2. Make a new ebs volume of that snapshot and put it on the zone where the ec2 instance lives.
3. Create an identical temporary new ec2 instance on the same zone.
4. Attach the snapshot volume you created on step 2 to the new instance.
5. ssh to the new machine.
6. sudo fdisk -l, you should see all the attached devices, you will see something like this referring to the attached ebs

Disk /dev/xvdf: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders, total 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdf doesn't contain a valid partition table

Don’t listen to that last message, you do have a valid partition.

7. Create a folder where to mount the disk. sudo mkdir /mnt/old-volume
8. Mount it sudo mount -t auto /dev/xvdf /mnt/old-volume
9. Get into /mnt/old-volume/etc/fstab and fix it.
10. Unmount /mnt/old-volume, turn off the instance, detach the repaired volume.
11. Turn off the original instance, detach the broken root volume (at /dev/sda1)
12. Attach the repaired volume to the original instance under /dev/sda1
13. Start the original instance.
14. ssh to it. (it will have a new ip address, make sure to update your DNS or load balancing entries)
15. Terminate the temporary instance and all the volumes that you won’t need.
16. Get to work.
17. Leave a tip below. 😉

Get FrostWire 1.0 for Android

If you are getting an Android device this holiday season, we’ve spent the last 5 months building a must-have app for it.

Install FrostWire via Google Play.

Download the .apk installer.

Here’s a few things you can do with our completely re-designed and easy to use file sharing client and media player.

Improved search that integrates with the cloud.

Search results are faster than ever (instant in most cases) and not only you’ll have the most powerful BitTorrent Deep-Smart search[1] featured in previous versions, now FrostWire integrates with popular cloud services where content creators from all over the world share millions of legally available files[2].

The fastest downloads ever on any FrostWire for Android.

Be it on Wi-Fi, 3G or 4G you will experience in most cases downloads that are so fast you won’t even see the progress bar begin to move, by the time the transfer screen is opened the file will already be on your device. Specially when sharing files with other FrostWire devices (PCs too) on the same Wi-Fi network.

Share files between phones, tablets and computers without any networking know-how

You don’t have to use cables or become a computer guru to share files between your phones, tablets or computers. We’ve also been working on FrostWire for desktop so that when you open FrostWire on your phone or computer you will be able to see every device automatically.

Just make files publicly available by opening padlock icons next to the files and every device running FrostWire will be able to download and even stream music or video from each other.

This makes it great for groups of people (work, school, home, party), now you only need to take only one photo and share it with everybody else next time you’re all connected to the same Wi-Fi hotspot.

Sending documents, music, family videos between your mobile, tablet or computer has never been easier or faster, best of all it doesn’t even matter what Operating system you’re running, Windows, Mac, Linux, it doesn’t matter, all you need is FrostWire and a Wi-Fi router. It just works.

World class Gesture Based Music Player

We’ve redesigned our music player for simplicity and convenience. FrostWire’s music player is the world’s first gesture based music player, we built it thinking of drivers in mind, no need to take your eyes off the road if you want to switch songs or pause/resume playback.

On this version we’ve introduced a beautiful new design with a 3D presentation of the album art currently being played. It also has functionality to share or un-share the file you’re currently listening to without having to browse or find the file on your device.

Easier Navigation

We’ve recreated the navigation system on the application, it now has a beautiful animated and responsive slide in menu that will take you anywhere in the app, with room for future functionality, and if you’re playing a song you will be able to see the album art of the song being played.

We’ve also implemented a custom “Back” history handler so that you can backtrack perfectly the screens you’ve navigated as you used the app. You now have random access navigation through the menu, and sequential access navigation through the Android’s back key.

We’ve added new controls to let you instantly filter your own files and see which are being shared on the Wi-Fi network and which aren’t in just one touch.

Install FrostWire on your Android and spread the word, the more the merrier, it’s absolutely free.

Install FrostWire via Google Play.

Download the .apk installer.

[1] FrostWire's Deep-Smart search uses the most popular torrent indexing services to fetch torrent files, then FrostWire opens these files and searches for the individual files indexed by the torrent. As FrostWire searches it remembers the files it's seen in the past to make your next searches instantaneous.

[2] Only on YouTube.com in 2012 it was announced users had shared over 4 million Creative Commons licensed videos. FrostWire also taps on SoundCloud.com, the internet's biggest sound and music community of content creators, bands, singers and DJs that pay to make their music available for free to the SoundCloud community. If you share your content on YouTube or SoundCloud you're automatically available to the millions of FrostWire users on both Desktop and Android. Integration with more popular cloud services to come in the next releases.

When you download and install FrostWire you must agree that you will not use it for the purpose of copyright infringement otherwise the software won't work. FrostWire condemns copyright infringement. Downloading FrostWire does not constitute permission or a license for obtaining or distributing unauthorized files. It is illegal for you to distribute copyrighted files without permission. If you want to know about legal content you can download and distribute legally please visit FrostClick.com, Vodo.net, ClearBits.net, LegitTorrents.info and CreativeCommons.org

OnLive could change the video game industry

I feel it’s my geek given duty to make a post about this presentation. I was lucky to finally have the time to watch their hour long presentation and Q&A session at the Game Developer Conference 2009 (which ends a couple of days from today). They could have not picked a better place to finally demo their technology.

In short, they’ve introduced a huge new concept to the video game industry, I’d call it “Cloud Gaming” to not only host the games, but also host the processing juice. You won’t need a console anymore, they keep the hardware to execute and stream the game to your screen. They support TV (with a miniconsole), PC and Mac.

So bear with me, they say they have solved the issue that you’re thinking about now, Lag. The people behind this worked on apple to create Quicktime, and they identified differences between what it takes to compress linear (regular) video, vs Interactive Video. They say their compression algorithm doesn’t take seconds of lag (like when you stream over a webcam), but miliseconds. They have custom chips to process the graphics, and I bet they might even built their own network protocol right on top of IP.

So what are some of the implications of this:

  • We’ll all be able to finally play Crysis and even more demanding games on low end PCs
  • No more buying more hardware, no more upgrading your PC to be able to run games, no more buying consoles
  • All your games live on the platform, so you can play from any computer, and you’ll keep the state of your game until the last time you hit the Pause button
  • Your friends can see you play, live. I bet we’ll be able to see live tournaments, we’ll start seeing a new breed of famous people get more attention, the Elite gamers. Imagine seeing the best Call of Duty player in the world playing live
  • New Genres of video games will emerge on this platform, maybe even new genres of entertainment, think new Live Broadcast shows where participants use an avatar to either act or compete (game show)
  • Game Developers not need to think of the rendering limitations that they might have nowadays, and will be able to design games that could only be imagined in the past. Render quality only thought for movies will now exist for video games, think of virtual reality now
  • There’s about 100 million PCs/Macs/Laptops out there that are not ready today to play high end games, now they’ll have the possibility of playing virtually any game by installing a 1Mb plugin from OnLive.com
  • Takes Piracy out of the Business Equation
  • A bigger gaming audience makes an even better case for companies placing advertisement in video games, maybe there will be a lot more high end free games with bigger audiences, think the next Grand Theft Auto coming out for free with superb real life like graphics rendering, all ad sponsored and free to the consumer. The amount of people that you could have playing a great game for free would make other developers think twice about charging for their games and having their virtual worlds ad sponsored.
  • No more installs
  • Now multiplayer will have almost no latency since all players live inside their datacenter, you only get the latency of your ISP if there’s any
  • Nintendo, Sony and Microsoft must be shitting their pants
  • Services that sell used games are going to be selling vintage and their business will be reduced

However I think there will always be room for the old consoles. This is the biggest entertainment industry in the world, we have grown up with consoles for almost 30 years and there’s a lot of changes to push into people’s minds:

  • How do you convince me, that finally made up my mind after years and dropped $500 on a PS3 to play with my friends in latin america online to switch to this, if my friends will probably have no way to even have access to the system in years to come?
  • How do you convince PC gamers that rather pay for the hardware and pirate the games? (There’s plenty of those, probably the majority of the PC gaming population outside the US never plays for a PC game, and doesn’t get into consoles because they pirate the games) into renting or buying games on the cloud?
  • How do you convince all the people that they should switch when their gaming experience depends entirely on being connected to the internet. So If the ISP is having issues I can’t play? isn’t my console awesome?

It seems that many of these complains are similar to all the complains brought upon business models that didn’t exist online and that are now thriving. This presentation left me with my mouth wide open, and I highly recommend you watch it. You’ll be blown away by the power of the UI, and how as you Browse for games, you can even see how other people are playing live, it’s like streaming video is nothing for OnLive. Really sick technology.

In the case that they succeed, I just can’t wait for them to have competition by the existing big brands, it’s going to get so interesting once cloud gaming becomes the defacto platform, maybe we as consumers will end up playing games for free, all sponsored with in game ads.

Just by listening to the guy if you’re a techie, your mind will start to fly to barely start to imagine the awesomeness of the technology that should be behind this. I can imagine anything from custom virtualization technology, to custom GPUs, custom network cards, custom network protocols on top of IP, deals with major internet backbone networks, ISPs, deals with game publishers, sick level API development, incomprehensible comprehension technology for my retard brain… when I see shit like this, I always think… how the hell is there people that still believe in god? Mankind is the closest thing there is to something like that! I’m thankful for people in this world that can think so big.

The service is supposed to launch next Winter 2009, but you can sign up to be a beta tester