Archive for the 'Code' Category

Setting up Eclipse as your IDE for Bitcoin C++ development on MacOSX.

Sunday, February 9th, 2014

If you are a Java developer used to the productivity levels achieved by working with eclipse’s code navigation, code completion and refactoring tools, it’s worth your time staying in eclipse for any sort of C++ development.

Screen Shot 2014-02-09 at 1.03.21 PM

This post refers specifically to getting your eclipse environment to work with a particular C++ Open Source project, The Bitcoin Project.

Before you start setting up eclipse, please make sure you can build Bitcoin from the command line, this way you know that you have everything necessary to build Bitcoin, even if you’re still getting a few errors showing in Eclipse, in the end Eclipse will be using the Makefiles provided by the project whenever we need to compile (and it can do so incrementally when possible saving you a lot of compilation time)

I’m assuming you have installed:
- eclipse
- eclipse CDT tools, up to date for the version of eclipse you’re working with (I’m still working with Juno)
- Qt/Eclipse plugin (optionally)
- All the dependencies (autoconf automake berkeley-db4 boost miniupnpc openssl pkg-config protobuf qt gdb) necessary to build Bitcoin which are easily installable via HomeBrew.

1. Let’s import the bitcoin/ project to our workspace.

File > Import > Existing Code as Makefile Project

Screen Shot 2014-02-09 at 12.41.45 PM

 

Look for the bitcoin/ git checkout folder, and make sure you use the GNU Autotools Toolchain

Screen Shot 2014-02-09 at 12.43.05 PM

 

Click Finish.

2. Fixing the C++ compiler Path and Symbols.

Right click on the project containing folder in the Project Explorer > Properties.
Go to C/C++ General > Paths and Symbols > Languages: GNU C++ >  ”Includes” Tab and make sure it looks something like the screenshot below (I got those paths by looking at  the  ones used by the Makefiles in the Bitcoin. Hit Apply , OK, then wait for the reindexing, you might still have a few weird errors because of how the compiler checking settings are.

Screen Shot 2014-02-09 at 12.55.38 PM

3. Remove a few more issues like “Error: Invalid arguments candidates are: void resize(?, int)."

We open again the project Properties, this time we go to C/C++ General > Preproessor Include Paths, Macros, etc.
Click on the Providers tab and make sure “CDT GCC Built-in Compiler Settings [Shared]” is checked. Hit Apply, OK, wait for reindexing.
If there are still errors, you might want to just delete them and refresh the project (F5 on the project folder in the Project explorer), all errors should be gone by now.

Screen Shot 2014-02-09 at 1.01.25 PM

Now start working just as fast as you’re used to with Java on Eclipse.

 

Code completion…

Screen Shot 2014-02-09 at 1.09.39 PM

 

Project wide renaming refactors in seconds…

Screen Shot 2014-02-09 at 1.10.07 PM

 

 

Find references of variables, methods, classes (Cmd+Shift+G)

Screen Shot 2014-02-09 at 1.23.37 PM

 

Find all the implementations of an interface (Cmd+T)

Screen Shot 2014-02-09 at 3.34.39 PM

and best of all

Interactive debugging with gdb*

Screen Shot 2014-02-11 at 1.21.07 PM

and all the tools you know and love from Eclipse.

*Setting up GDB debugging

To do step by step debugging you can use gdb, if you don’t have it installed just go to your Terminal and type brew install gdb.

On your command line, execute your Makefile to create an executable, once it appears on your Project Explorer you can Right click on it Debug As > Debug Configuration…

Screen Shot 2014-02-11 at 1.31.43 PM

then make sure you have set gdb as the executable debugger in the “Debugger” configuration tab, then just set your breakpoints and debug away!

Screen Shot 2014-02-11 at 1.32.00 PM

Not so fast… :(

As of Mac OSX 10.9, Apple decided that you cannot use gdb unless the gdb executable is signed with a certificate, they want you to use their super duper lldb debugger, but it’s still not compatible with Eclipse, you know, so you use their XCode IDE instead of what you want to use…

Anyways, signing the gdb at /usr/local/bin/gdb is not that hard.

To sign it you can create a certificate, or use an existing developer certificate. In my case, I already had a Mac Developer certificate so it was a very simple process, just issuing a single command in the Terminal and I finally got rid of the "Unable to find Mach task port for process-id 93213: (os/kern) failure (0x5).\n (please check gdb is codesigned - see taskgated(8))" error.

codesign -s “Name of my certificate here” /usr/local/bin/gdb

Then I tried debugging, I got a password dialog to verify I was the owner of the certificate, and then gdb could take over and then I could do my step by step debugging, with the ocassional crash.

Happy Hacking.

How to actually build bitcoin on Mac OSX 10.9.1

Monday, February 3rd, 2014

First of all, if you have Macports, do yourself a favor and get rid of it.

Then make sure you have Homebrew installed and all the packages installed by it up to date.

1. Let’s install all the dependencies for Bitcoin hacking.

brew install autoconf automake berkeley-db4 boost miniupnpc openssl pkg-config protobuf qt

2. Make sure you have the right OpenSSL version installed. Type the following on your terminal:

openssl version

you should see “OpenSSL 1.0.1f 6 Jan 2014.”

if you see an older version, do

brew update
brew upgrade

OpenSSL should be upgraded, you may or may not have to issue a “brew link openssl” or even a “brew link --overwrite openssl” if it’s giving you trouble.

3. Now, let’s configure, and make. I strongly suggest you add the boost library path when configuring, otherwise you may get nasty “Undefined symbols for architecture x86_64” compilation errors. During the time I wrote this, homebrew had installed boost 1.55 in my system, and the boost lib path was /usr/local/Cellar/boost/1.55.0/lib so I invoked the following:

./configure --with-boost-libdir=/usr/local/Cellar/boost/1.55.0/lib

After that I just issued a

make

And I was done.

If you want to hack the bitcoin-qt client like me, head to src/qt/, there should be a bitcoin-qt executable there now.

Enjoy

building cgminer from source on OSX

Sunday, January 19th, 2014

so you cloned the cgminer repo from github to build on your OSX machine and you get this bullshit error

$ ./autogen.sh
readlink: illegal option -- f
usage: readlink [-n] [file ...]
usage: dirname path
touch: /ltmain.sh: Permission denied
Use of chdir('') or chdir(undef) as chdir() is deprecated at /usr/local/bin/autoreconf line 670.
Configuring...
./autogen.sh: line 10: /configure: No such file or directory

readlink works differently in OSX and the current version of the autogen.sh script seems like it wasn’t tested on OSX (wonder why didn’t they use a simple bs_dir=`pwd`, the answer is probably canonical paths and what not).

To keep moving along, open the autogen.sh script and just change the value of the bs_dir variable to the full real path of where you have cloned the cgminer source code.

then execute your autogen script, make sure to enable compilation flags for your ASIC hardware, in my case I remember seeing ‘icarus’ on a binary build of cgminer I tried before, so I did

./autogen.sh --enable-icarus

you might want to enable all of them if you’re not sure what hardware you have or you will have in the future as you may not like the joys of building software (check out the the README for all the –enable-xxx options available)

If you’re getting errors on your configuration script due to missing dependencies, I strongly recommend you use Homebrew to install these packages (if you are using Macports or Fink, I strongly suggest you completely remove that crap from your computer and go 100% with brew, it works really well if you’re building a lot of code on a regular basis):

brew install autoconf automake autoreconf libtool openssl curses curl

brew, at the point of this writing didn’t have libcurl, so that one you will have to download, ./configure, make and sudo make install yourself from here http://curl.haxx.se/download.html (I used version 7.34 when I did it)

after that the autogen script should work, and then you should be just one ‘make’ away from your goal.

How to have a Play framework app autostart during boot on Elastic Beanstalk CentOS ec2 instances

Friday, September 27th, 2013

So you’ve created an Elastic Beanstalk environment, you have a play framework distribution which you’ve created using play dist (either on your local environment, or right there on the server, whatever you prefer)

play dist outputs a my-app-1.0.zip file which has a self-contained version of your app with all the necessary libraries and a start script.

Afer you unzip it, you end up with a my-app-1.0/lib/ folder and a start script.

[ec2-user@ip-10-235-8-106 bullq-1.0]$ ls -l
total 24
drwxrwxr-x 2 ec2-user ec2-user 4096 Sep 27 15:35 lib
-rwxrwxr-x 1 ec2-user ec2-user 4328 Sep 27 15:35 start

Make sure it’s executable by using chmod +x start on the start script.

So now, this is all in the first ec2 instance of your elastic beanstalk environment, if you’re like me and you’ve used ubuntu/debian for your server management things can be slightly different here, since Amazon preferred CentOS for their default image, and here I’ll show you how to make your play app auto start when the server boots because you want every new machine that may be instanciated to have your app installed and to start the service as soon as the machine is up.

Create a /etc/init.d/myappd script
(I’m using ‘myapp’ here as an example, your app can be named whatever is named, so replace accordingly)

#!/usr/bin/env bash
# myappd
# Script to start|stop|restart myappd from /etc/init.d/
# By Gubatron - @gubatron - gubatron@gmail.com

#replace accordingly in these variables 'myapp' for the name of your app
PID_FILE=/home/ec2-user/myapp/dist/myapp-1.0/RUNNING_PID
DAEMON_NAME=myappd
DAEMON_PATH=/home/ec2-user/myapp
DAEMON=$DAEMON_PATH/dist/myapp-1.0/start

test -x $DAEMON || exit 0

set -e
 
function killDAEMON() {
  echo "start kill daemon"
  kill -9 `cat /home/ec2-user/bullq/dist/bullq-1.0/RUNNING_PID`
  echo "end kill daemon"
}

function removePIDFile() {
  if [ -e $PID_FILE ]
  then
    rm -f $PID_FILE
  fi
}
 
case $1 in
  start)
        removePIDFile
        echo "Starting $DAEMON_NAME... $DAEMON"
        nohup $DAEMON &
        ;;
  restart)
        echo "Hot restart of $DAEMON_NAME"
        killDAEMON
        removePIDFile
        COMMAND="nohup $DAEMON &";
        echo $COMMAND
        `$COMMAND`
        rm -f $PID_FILE
        ;;
  stop)
        echo "Stopping $DAEMON_NAME"
        killDAEMON
        removePIDFile
        ;;
  *)
        echo "Usage: $DAEMON_NAME {start|restart|stop}" >&2
        exit 1
        ;;
esac
 
exit 0

Save that script, and now you’re able to execute the following commans to start, stop and restart your play app.

/etc/init.d/myappd start
/etc/init.d/myappd stop
/etc/init.d/myappd restart

Wire it to autostart

The simplest way I found to have this script start when the server would boot was to add it at the end of the
/etc/rc.local file. (In ubuntu you’d register the new script with the upate-rc.d command)

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

/etc/init.d/myappd start

How to add an existing GIT repository to github.

Saturday, August 31st, 2013

Most of the times, it makes more sense to start working on something that slowly transforms into the beginning of a project that deserves to be on github. This post is about creating a local repository and putting it on github.

1. First we must convert the main local folder into a git repository. For this example let’s call the folder “my-new-project”. With your terminal go to that folder and type:

git init

the repository will initialize but nothing will be added to it yet. If you type git status you will see all the things you can add to it, so use git add to add the folders you want to track, and then go ahead and do a git commit -m “initial commit”

2. Now go to github.com and create your repository “my-new-project”, and copy the clone url of the repo, I personally like to work with the one that starts with “ssh://” since I like to work with ssh keys and not have to deal with passwords.

You can easily configure your ssh certificates for multiple things, not just github but keys for many many servers working with the ~/.ssh/config file (no need to deal with effing ssh-agent).

If you created this github repository with some other account, make sure to give yourself contributor access on the github role settings, otherwise you won’t be able to pull/push.

3. Time to pull (fetch+merge) the remote repo and then push this baby up.
You do that by invoking the following commands (let’s suppose the remote url is git remote add origin git@github.com:myaccount/my-new-project.git):

git remote add git@github.com:myaccount/my-new-project.git
git pull origin master

you should see something like below coming from the remote repo’s master branch:

warning: no common commits
remote: Counting objects: 4, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 4 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (4/4), done.
From github.com:myaccount/my-new-project
 * branch            master     -> FETCH_HEAD
Merge made by the 'recursive' strategy.
 .gitignore | 6 ++++++
 README.md  | 2 ++
 2 files changed, 8 insertions(+)
 create mode 100644 .gitignore
 create mode 100644 README.md

and then just

git push

and you’re done, you should see your initial commit on github now.

can’t ssh to ec2 ubuntu instance, /etc/fstab breaks bootup due to missing ebs volume [SOLVED]

Wednesday, August 21st, 2013

Screen Shot 2013-08-21 at 12.08.04 PM

So the /etc/fstab file on your root volume looked like this

LABEL=cloudimg-rootfs / ext4 defaults 0 0
/dev/xvdf /mnt/backups auto defaults,comment=cloudconfig 0 2

by mistake you deleted the ebs volume that you had mounted on /mnt/backups (or whatever folder) and you restarted your ubuntu instance not knowing that if the /etc/fstab would break it would not continue to start all the application layer networking services like ssh on port 22…

you can ping the machine, but you can’t ssh, amazon support won’t respond or will tell you to fuck yourself.

you learn that ubuntu has had this bug for a while, but it’s been addressed by passing your volume configuration a nobootwait option.

you wish your /etc/fstab looked like this, but you can’t get in, amazon doesn’t give you any other options from their console to go in and solve the problem through a console…

LABEL=cloudimg-rootfs / ext4 defaults 0 0
/dev/xvdf /mnt/backups auto defaults,nobootwait,comment=cloudconfig 0 2

No worries, I have a fix that will let you edit that file, and boot back and try to recover things, you may have lost that ebs volume, but you won’t have to setup this computer again.

1. Make a snapshot of the root volume on that instance. This will take a while.
2. Make a new ebs volume of that snapshot and put it on the zone where the ec2 instance lives.
3. Create an identical temporary new ec2 instance on the same zone.
4. Attach the snapshot volume you created on step 2 to the new instance.
5. ssh to the new machine.
6. sudo fdisk -l, you should see all the attached devices, you will see something like this referring to the attached ebs

Disk /dev/xvdf: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders, total 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvdf doesn't contain a valid partition table

Don’t listen to that last message, you do have a valid partition.

7. Create a folder where to mount the disk. sudo mkdir /mnt/old-volume
8. Mount it sudo mount -t auto /dev/xvdf /mnt/old-volume
9. Get into /mnt/old-volume/etc/fstab and fix it.
10. Unmount /mnt/old-volume, turn off the instance, detach the repaired volume.
11. Turn off the original instance, detach the broken root volume (at /dev/sda1)
12. Attach the repaired volume to the original instance under /dev/sda1
13. Start the original instance.
14. ssh to it. (it will have a new ip address, make sure to update your DNS or load balancing entries)
15. Terminate the temporary instance and all the volumes that you won’t need.
16. Get to work.
17. Leave a tip below. ;)

[SOLVED] Java7 SMTP Issue (Caused by: sun.security.pkcs11.wrapper.PKCS11Exception)

Wednesday, August 7th, 2013

So you had your little program that would use AWS to send emails, and all of a sudden after a Java 7 update you get a stack trace like this:

javax.mail.MessagingException: Could not connect to SMTP host: email-smtp.us-east-1.amazonaws.com, port: 465;
  nested exception is:
	javax.net.ssl.SSLException: Server key
	at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1962)
	at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:654)
	at javax.mail.Service.connect(Service.java:295)
	at javax.mail.Service.connect(Service.java:176)
	at javax.mail.Service.connect(Service.java:196)
...
Caused by: javax.net.ssl.SSLException: Server key
	at sun.security.ssl.Handshaker.throwSSLException(Handshaker.java:1274)
...
Caused by: java.security.spec.InvalidKeySpecException: Could not create EC public key
	at sun.security.pkcs11.P11ECKeyFactory.engineGeneratePublic(P11ECKeyFactory.java:169)
	at java.security.KeyFactory.generatePublic(KeyFactory.java:334)
...
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_DOMAIN_PARAMS_INVALID
	at sun.security.pkcs11.wrapper.PKCS11.C_CreateObject(Native Method)
	at sun.security.pkcs11.P11ECKeyFactory.generatePublic(P11ECKeyFactory.java:233)

It seems like the SSL Socket Factory in Java 7 is missing some of the Ciphers when it comes to setting up SSL socket cipher suits (see SSLSocket.setEnabledCipherSuites)

so… the question is now,
how do I tell java mail to use the cipher suit I used to use in java 6?

Easy, when you’re setting up your Session properties, make sure to include the following key:

properties.put("mail.smtp.ssl.ciphersuites","SSL_RSA_WITH_RC4_128_MD5 SSL_RSA_WITH_RC4_128_SHA TLS_RSA_WITH_AES_128_CBC_SHA TLS_DHE_RSA_WITH_AES_128_CBC_SHA TLS_DHE_DSS_WITH_AES_128_CBC_SHA SSL_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA SSL_RSA_WITH_DES_CBC_SHA SSL_DHE_RSA_WITH_DES_CBC_SHA SSL_DHE_DSS_WITH_DES_CBC_SHA SSL_RSA_EXPORT_WITH_RC4_40_MD5 SSL_RSA_EXPORT_WITH_DES40_CBC_SHA SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA TLS_EMPTY_RENEGOTIATION_INFO_SCSV");

which specifies in a space separated String a list of tokens that represent the cipher suites the SSL sockets will use. At some point that list gets split into a String[] and passed to SSLSocket.setEnabledCipherSuites.

This is how my session properties look like in case you want to know:

        Properties properties = new Properties();
        properties.put("mail.transport.protocol", "smtp");
        properties.put("mail.smtp.auth", true); 
        properties.put("mail.smtp.ssl.enable",true);
        properties.put("mail.smtp.port", emailSMTPPort);
        properties.put("mail.smtp.host", emailSMTPHost);
        properties.put("mail.smtp.ssl.ciphersuites","SSL_RSA_WITH_RC4_128_MD5 SSL_RSA_WITH_RC4_128_SHA TLS_RSA_WITH_AES_128_CBC_SHA TLS_DHE_RSA_WITH_AES_128_CBC_SHA TLS_DHE_DSS_WITH_AES_128_CBC_SHA SSL_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA SSL_RSA_WITH_DES_CBC_SHA SSL_DHE_RSA_WITH_DES_CBC_SHA SSL_DHE_DSS_WITH_DES_CBC_SHA SSL_RSA_EXPORT_WITH_RC4_40_MD5 SSL_RSA_EXPORT_WITH_DES40_CBC_SHA SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA TLS_EMPTY_RENEGOTIATION_INFO_SCSV");

Cheers, don’t forget to tip If I saved your ass.

Obama Warholized by #CODE on Williamsburg Bridge

Wednesday, August 7th, 2013

obama-warholized-by-code-williamsburg-bridge

Obama Warholized by #CODE (Williamsburg, Brooklyn)

Wednesday, August 7th, 2013

obama-warholized-by-code-in-random-williamsburg-building

VIDEO: Coding the FrostWire search filters UI

Monday, June 24th, 2013

Get in the zone with me for a good 15 minutes, maybe you’ll catch a few eclipse tricks and you’ll learn a little bit about how I think (and make mistakes along the way of fixing something on FrostWire)




  • Categories

  • April 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • June 2011
  • May 2011
  • April 2011
  • March 2011
  • February 2011
  • December 2010
  • November 2010
  • October 2010
  • September 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • October 2009
  • September 2009
  • July 2009
  • May 2009
  • April 2009
  • March 2009
  • February 2009
  • January 2009
  • December 2008
  • November 2008
  • October 2008
  • September 2008
  • August 2008
  • July 2008
  • June 2008
  • May 2008
  • April 2008
  • March 2008
  • February 2008
  • January 2008
  • December 2007
  • November 2007
  • October 2007
  • September 2007
  • August 2007
  • July 2007
  • June 2007
  • May 2007
  • April 2007
  • March 2007
  • February 2007
  • January 2007
  • December 2006
  • November 2006
  • October 2006
  • September 2006
  • August 2006
  • July 2006
  • June 2006
  • May 2006
  • April 2006
  • March 2006
  • February 2006
  • January 2006
  • December 2005
  • November 2005
  • October 2005
  • September 2005
  • August 2005
  • July 2005
  • June 2005
  • May 2005
  • April 2005
  • March 2005
  • February 2005
  • January 2005
  • December 2004
  • November 2004
  • October 2004