[TUTORIAL] HOW TO PAUSE AN AWS ELASTIC BEANSTALK ENVIRONMENT USING THE DASHBOARD

So you have set up a test elastic beanstalk environment because you don’t want to put at risk your production elastic bean.
You don’t want to pay for the time you’re not using it, and there’s no freaking “Pause Environment” option.

aws-elasticbean-no-pause-option

all you can do is “Terminate Environment”? but then to crete the damn environment it takes forever to configure (if you’re using the dashboard), so we don’t want to do that.

If you try to terminate the associated EC2 instance, the Elastic Beanstalk environment will automatically provision a new machine for you, so that doesn’t work. What do you do?

Open the Elastic Bean environment you want to pause and go to “Configuration”
Screen Shot 2016-06-23 at 1.55.02 PM

On the Scaling card, click on the Gear icon to change its settings

Screen Shot 2016-06-23 at 1.55.11 PM

in the “Auto Scaling” options, set the “Minimum instance count” to 0, and the “Maximum instance count” to 0

Screen Shot 2016-06-23 at 1.55.56 PM

Hit “Apply”
Screen Shot 2016-06-23 at 1.56.03 PM

Wait for your environment to apply the new configuration

Screen Shot 2016-06-23 at 1.56.15 PM

 

Now your health status has switched to gray. It’s now paused and you won’t be paying for the test ec2 instance while you’re not working on it.

Screen Shot 2016-06-23 at 1.58.11 PM

If you are curious and you check the ec2 instances associated to the environment, it/them should be terminated now, but now the environment won’t try to provision new ones.
thanks-bow

Savings in your pocket. Cheers.

 

command line speed test, see how fast is your server’s connection

Save the following script in a file called speed_test

#!/bin/bash

# Requirements
# sudo apt-get install lftp iperf

lftp -e 'pget http://releases.ubuntu.com/14.04.3/ubuntu-14.04.3-desktop-amd64.iso; exit; '

make sure the file is executable: sudo chmod +x speed_test

Once you have installed lftp and iperf make sure you have the script somewhere in your $PATH.

The script basically downloads an ubuntu iso and does the math.

The output looks like this on a AWS m3.large instance:

$ speed_test
1054871586 bytes transferred in 14 seconds (70.37M/s)

Multiply by 8 to convert 70.37M/s to Megabits per second = 562.96 Mbit/s

AWS’s download speeds for m3.large instances is half a gigabit in January 2016. (or is that the upload speed of the Ubuntu ISO server?)

How to resize an EBS (xfs formatted) partition

First of all, create a snapshot of your EBS volume. Then out of that snapshot you will be able to create your new volume.

However, when you detach the old one from your instance and attach the new one, you will still see the old available space with df

look at my /dev/xvdf/ available space (after mounting the new EBS volume)

ubuntu@ip-10-47-167-74:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       16G  5.9G  9.1G  40% /
udev            7.4G   12K  7.4G   1% /dev
tmpfs           1.5G  176K  1.5G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.4G     0  7.4G   0% /run/shm
/dev/xvdf        20G   19G  1.7G  92% /media/ebs/data  <<< this one

still 20G, I mounted a 80G one!

This drive has been formatted to use an xfs file system. In order to resize it, this is the command I used:

sudo xfs_growfs -d /media/ebs/data

output should be something like this

ubuntu@ip-10-47-167-74:~$ sudo xfs_growfs -d /media/ebs/data
meta-data=/dev/xvdf              isize=256    agcount=4, agsize=1310720 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=5242880, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
data blocks changed from 5242880 to 20971520

now let’s see the df -h output

ubuntu@ip-10-47-167-74:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       16G  5.9G  9.1G  40% /
udev            7.4G   12K  7.4G   1% /dev
tmpfs           1.5G  176K  1.5G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.4G     0  7.4G   0% /run/shm
/dev/xvdf        80G   19G   62G  23% /media/ebs/data

How to have a Play framework app autostart during boot on Elastic Beanstalk CentOS ec2 instances

So you’ve created an Elastic Beanstalk environment, you have a play framework distribution which you’ve created using play dist (either on your local environment, or right there on the server, whatever you prefer)

play dist outputs a my-app-1.0.zip file which has a self-contained version of your app with all the necessary libraries and a start script.

Afer you unzip it, you end up with a my-app-1.0/lib/ folder and a start script.

[ec2-user@ip-10-235-8-106 bullq-1.0]$ ls -l
total 24
drwxrwxr-x 2 ec2-user ec2-user 4096 Sep 27 15:35 lib
-rwxrwxr-x 1 ec2-user ec2-user 4328 Sep 27 15:35 start

Make sure it’s executable by using chmod +x start on the start script.

So now, this is all in the first ec2 instance of your elastic beanstalk environment, if you’re like me and you’ve used ubuntu/debian for your server management things can be slightly different here, since Amazon preferred CentOS for their default image, and here I’ll show you how to make your play app auto start when the server boots because you want every new machine that may be instanciated to have your app installed and to start the service as soon as the machine is up.

Create a /etc/init.d/myappd script
(I’m using ‘myapp’ here as an example, your app can be named whatever is named, so replace accordingly)

#!/usr/bin/env bash
#myappd
#Script to start|stop|restart myappd from /etc/init.d/
#By Gubatron – @gubatron – gubatron@gmail.com

#replace accordingly in these variables ‘myapp’ for the name of your app
PID_FILE=/home/ec2-user/myapp/dist/myapp-1.0/RUNNING_PID
DAEMON_NAME=myappd
DAEMON_PATH=/home/ec2-user/myapp
DAEMON=$DAEMON_PATH/dist/myapp-1.0/start

test -x $DAEMON || exit 0

set -e

function killDAEMON() {
echo “start kill daemon”
kill -9 cat /home/ec2-user/bullq/dist/bullq-1.0/RUNNING_PID
echo “end kill daemon”
}

function removePIDFile() {
if [ -e $PID_FILE ]
then
rm -f $PID_FILE
fi
}

case $1 in
start)
removePIDFile
echo “Starting $DAEMON_NAME… $DAEMON”
nohup $DAEMON &
;;
restart)
echo “Hot restart of $DAEMON_NAME”
killDAEMON
removePIDFile
COMMAND=”nohup $DAEMON &”;
echo $COMMAND
$COMMAND
rm -f $PID_FILE
;;
stop)
echo “Stopping $DAEMON_NAME”
killDAEMON
removePIDFile
;;
*)
echo “Usage: $DAEMON_NAME {start|restart|stop}” >&2
exit 1
;;
esac

exit 0

 

Wire it to autostart

The simplest way I found to have this script start when the server would boot was to add it at the end of the
/etc/rc.local file. (In ubuntu you’d register the new script with the upate-rc.d command)

#!/bin/sh
#
This script will be executed after all the other init scripts.
You can put your own initialization stuff in here if you don’t
want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

/etc/init.d/myappd start

 

can’t ssh to ec2 ubuntu instance, /etc/fstab breaks bootup due to missing ebs volume [SOLVED]

Screen Shot 2013-08-21 at 12.08.04 PM

So the /etc/fstab file on your root volume looked like this

LABEL=cloudimg-rootfs / ext4 defaults 0 0
/dev/xvdf /mnt/backups auto defaults,comment=cloudconfig 0 2

by mistake you deleted the ebs volume that you had mounted on /mnt/backups (or whatever folder) and you restarted your ubuntu instance not knowing that if the /etc/fstab would break it would not continue to start all the application layer networking services like ssh on port 22…

you can ping the machine, but you can’t ssh, amazon support won’t respond or will tell you to fuck yourself.

you learn that ubuntu has had this bug for a while, but it’s been addressed by passing your volume configuration a nobootwait option.

you wish your /etc/fstab looked like this, but you can’t get in, amazon doesn’t give you any other options from their console to go in and solve the problem through a console…

LABEL=cloudimg-rootfs / ext4 defaults 0 0
/dev/xvdf /mnt/backups auto defaults,nobootwait,comment=cloudconfig 0 2

No worries, I have a fix that will let you edit that file, and boot back and try to recover things, you may have lost that ebs volume, but you won’t have to setup this computer again.

1. Make a snapshot of the root volume on that instance. This will take a while.
2. Make a new ebs volume of that snapshot and put it on the zone where the ec2 instance lives.
3. Create an identical temporary new ec2 instance on the same zone.
4. Attach the snapshot volume you created on step 2 to the new instance.
5. ssh to the new machine.
6. sudo fdisk -l, you should see all the attached devices, you will see something like this referring to the attached ebs

Disk /dev/xvdf: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders, total 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdf doesn't contain a valid partition table

Don’t listen to that last message, you do have a valid partition.

7. Create a folder where to mount the disk. sudo mkdir /mnt/old-volume
8. Mount it sudo mount -t auto /dev/xvdf /mnt/old-volume
9. Get into /mnt/old-volume/etc/fstab and fix it.
10. Unmount /mnt/old-volume, turn off the instance, detach the repaired volume.
11. Turn off the original instance, detach the broken root volume (at /dev/sda1)
12. Attach the repaired volume to the original instance under /dev/sda1
13. Start the original instance.
14. ssh to it. (it will have a new ip address, make sure to update your DNS or load balancing entries)
15. Terminate the temporary instance and all the volumes that you won’t need.
16. Get to work.
17. Leave a tip below. 😉

[SOLVED] Java7 SMTP Issue (Caused by: sun.security.pkcs11.wrapper.PKCS11Exception)

So you had your little program that would use AWS to send emails, and all of a sudden after a Java 7 update you get a stack trace like this:

[java]
javax.mail.MessagingException: Could not connect to SMTP host: email-smtp.us-east-1.amazonaws.com, port: 465;
nested exception is:
javax.net.ssl.SSLException: Server key
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1962)
at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:654)
at javax.mail.Service.connect(Service.java:295)
at javax.mail.Service.connect(Service.java:176)
at javax.mail.Service.connect(Service.java:196)

Caused by: javax.net.ssl.SSLException: Server key
at sun.security.ssl.Handshaker.throwSSLException(Handshaker.java:1274)

Caused by: java.security.spec.InvalidKeySpecException: Could not create EC public key
at sun.security.pkcs11.P11ECKeyFactory.engineGeneratePublic(P11ECKeyFactory.java:169)
at java.security.KeyFactory.generatePublic(KeyFactory.java:334)

Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_DOMAIN_PARAMS_INVALID
at sun.security.pkcs11.wrapper.PKCS11.C_CreateObject(Native Method)
at sun.security.pkcs11.P11ECKeyFactory.generatePublic(P11ECKeyFactory.java:233)
[/java]

It seems like the SSL Socket Factory in Java 7 is missing some of the Ciphers when it comes to setting up SSL socket cipher suits (see SSLSocket.setEnabledCipherSuites)

so… the question is now,
how do I tell java mail to use the cipher suit I used to use in java 6?

Easy, when you’re setting up your Session properties, make sure to include the following key:

[java]properties.put(“mail.smtp.ssl.ciphersuites”,”SSL_RSA_WITH_RC4_128_MD5 SSL_RSA_WITH_RC4_128_SHA TLS_RSA_WITH_AES_128_CBC_SHA TLS_DHE_RSA_WITH_AES_128_CBC_SHA TLS_DHE_DSS_WITH_AES_128_CBC_SHA SSL_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA SSL_RSA_WITH_DES_CBC_SHA SSL_DHE_RSA_WITH_DES_CBC_SHA SSL_DHE_DSS_WITH_DES_CBC_SHA SSL_RSA_EXPORT_WITH_RC4_40_MD5 SSL_RSA_EXPORT_WITH_DES40_CBC_SHA SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA TLS_EMPTY_RENEGOTIATION_INFO_SCSV”);[/java]

which specifies in a space separated String a list of tokens that represent the cipher suites the SSL sockets will use. At some point that list gets split into a String[] and passed to SSLSocket.setEnabledCipherSuites.

This is how my session properties look like in case you want to know:

[java]
Properties properties = new Properties();
properties.put(“mail.transport.protocol”, “smtp”);
properties.put(“mail.smtp.auth”, true);
properties.put(“mail.smtp.ssl.enable”,true);
properties.put(“mail.smtp.port”, emailSMTPPort);
properties.put(“mail.smtp.host”, emailSMTPHost);
properties.put(“mail.smtp.ssl.ciphersuites”,”SSL_RSA_WITH_RC4_128_MD5 SSL_RSA_WITH_RC4_128_SHA TLS_RSA_WITH_AES_128_CBC_SHA TLS_DHE_RSA_WITH_AES_128_CBC_SHA TLS_DHE_DSS_WITH_AES_128_CBC_SHA SSL_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA SSL_RSA_WITH_DES_CBC_SHA SSL_DHE_RSA_WITH_DES_CBC_SHA SSL_DHE_DSS_WITH_DES_CBC_SHA SSL_RSA_EXPORT_WITH_RC4_40_MD5 SSL_RSA_EXPORT_WITH_DES40_CBC_SHA SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA TLS_EMPTY_RENEGOTIATION_INFO_SCSV”);
[/java]

Cheers, don’t forget to tip If I saved your ass.

Deploying HTML5 apps on CloudFront with efficient invalidation requests

So you decided to build your next web app/site using nothing but HTML5 and Javascript.
No server side processing for anything related to UI.

This means you will be coding a lot of JavaScript.

Wouldn’t it be nice to put all that static HTML and JS on your CloudFront CDN and not deal with web servers?

Once you deploy it’s nice, but Amazon Cloudfront is not meant for you to be invalidating objects all the time, they actually charge you $0.005 per each invalidated file.

They tell you to version your files, but if you’re going for a no webserver based deployment, 100% cloudfront deployment, you can’t play around with url rewrites or redirects, so you’ll want to invalidate the files that have changed.

Here’s a script to invalidate only the files that have changed.

Our configuration
Cloudfront pointed to a custom origin.

The custom origin web server Document Root points to
/var/www/mysite/production

‘production’ is a symlink pointing to a folder inside /var/www/mysite, in there we have copies of the entire website saved under version numbers, for example

[bash]/var/www/mysite/1.0.1
/var/www/mysite/1.0.2
/var/www/mysite/1.0.3 -> production/
/var/www/mysite/1.0.4[/bash]

What our script does, is it uses rsync to compare the files that changed between the current production directory (in this case 1.0.3 with the next version 1.0.4), and dump the new files on a temporary folder /var/www/mysite/tmp

We then create a list with all the files in the /tmp folder, and for each 1,000 files that have changed we send Cloudfront an invalidation request and move the ‘production’ symlink to the new version.

Instead of invalidating every file (which would be costly), we just invalidate what changed, and we don’t have to change any of our code to reflect version numbers.

All of this is handled by command line tools that let us do this in one step (one step deployment)

Here’s some of the python code we use to get the list of files that changed in order to build the invalidation request with the minimum files necessary so Jeff Bezos doesn’t take all our money.

[python]
def getFilesToInvalidate(config, newVersion):
”’Compares the files in the new version folder with what’s currently in production.
returns a list of the files that changed or got removed.”’
checkKey(config,’website.exports.path’)

oldFiles = config[‘website.exports.path’] + os.path.sep + ‘production’
diffFiles = config[‘website.exports.path’] + os.path.sep + ‘diff’

#the ../NEW_VERSION is important, the command needs relative paths to work
#rsync -av –compare-dest=../NEW_VERSION /path/to/OLD_VERSION /path/to/TEMP_CHANGED
rmDiffCmd = ‘rm -fr ‘ + diffFiles
cmd = ‘rsync -av –compare-dest=../%s %s/ %s’ % (newVersion,
oldFiles,
diffFiles)
print cmd
os.system(rmDiffCmd)
os.system(cmd)
diffList = getFilesInPath(diffFiles)
result = []
for i in xrange(len(diffList)):
if diffList[i].find(‘#’)!=-1:
continue
result.append(diffList[i].replace(diffFiles,”))
os.system(rmDiffCmd)

return result
[/python]

Then use the ‘result’ list to create your CloudFront invalidation request.

Screw server side processing to render web pages, clients can do it now with client-side templating and javascript.
And oh Amazon, you guys need to step up your CloudFront Console features, it’d be awesome to invalidate paths and what not, it’s ultimately about what the customer needs.

Here’s toast to a faster cloud, cheers.