## Archive for the 'Code' Category

### # Things to remember when compiling/linking C/C++ software

Wednesday, March 18th, 2015

by Angel Leon. March 17, 2015.

## Include Paths

On the compilation phase, you will usually need to specify the different include paths so that the interfaces (.h, .hpp) which define structs, classes, constans, and functions can be found.

With gcc and llvm include paths are passed with -I/path/to/includes, you can pass as many -I as you need.

In Windows, cl.exe takes include paths with the following syntax:
/I"c:\path\to\includes\ you can also pass as many as you need.

Some software uses macro definition variables that should be passed during compile time to decide what code to include.

## Compilation flags

These compilation-time variables are passed using -D,
e.g. -DMYSOFTWARE_COMPILATION_VARIABLE -DDO_SOMETHING=1 DDISABLE_DEPRECATED_FUNCTIONS=0

These compilation time flags are by convention usually put into a single variable named CXXFLAGS, which is then passed to the compiler as a parameter for convenience when you’re building your compilation/make script.

## Object files

When you compile your .c, or .cpp files, you will end up with object files.
These files usually have .o extensions in Linux, in Windows they might be under .obj extensions.

You can create an .o file for a single or for many source files.

## Static Library files

When you have several .o files, you can put them together as a library, a static library. In Linux/Mac these static libraries are simply archive files, or .a files. In windows, static library files exist under the .lib extension.

They are created like this in Linux/Mac:

ar -cvq libctest.a ctest1.o ctest2.o ctest3.o

libctest.a will contain ctest1.o,ctest2.o and ctest2.o

They are created like this in Windows:

LIB.EXE /OUT:MYLIB.LIB FILE1.OBJ FILE2.OBJ FILE3.OBJ

When you are creating an executable that needs to make use of a library, if you use these static libraries, the size of your executable will be the sum of all the object files statically linked by the executable. The code is right there along the executable, it’s easier to distribute, but again, the size of the executable can be bigger than it needs to… why? because, sometimes, many of the .o files, or even the entire .a file you’re linking against might be a standard library that many other programs need.

## Shared Libraries (Dynamic Libraries)

So shared or dynamic libraries were invented so that different programs or libraries would make external (shared) references to them, since they’re “shared” the symbols defined in them don’t need to be part of your executable or library, your executable contain symbols whose entry points or offset addresses might point to somewhere within themselves, but they will also have symbols whose entry points are expected to exist on shared libraries which need only be loaded once in a single portion of the operating shared memory, thus not just making the size of your executable as small as it needs to be, but you won’t need to load the library for every process/program that needs its symbols.

On Linux shared files exist under the .so (shared object) file extension, on Mac .dylib (dynamic library), and in Windows they’re called .dll (dynamic link libraries)

Another cool thing about dynamic libraries, is that they can be linked during runtime, not just compile time. An example of runtime dynamic libraries are browser plugins.

In Linux .so files are created like this:

gcc -Wall -fPIC -c *.c
gcc -shared -Wl,-soname,libctest.so.1 -o libctest.so.1.0   *.o

• -Wall enables all warnings.
• -c means don’t run the linker.
• -fPIC means “Position Independent Code”, a requirement for shared libraries in Linux.
• -shared makes the object file created shareable by different executables.
• -Wl passes a comma separated list of arguments to the linker.
• -soname means “shared object name” to use.

In Mac .dylib files are created like this:

clang -dynamiclib -o libtest.dylib file1.o file2.o -L/some/library/path -lname_of_library_without_lib_prefix

In Windows .dll files are created like this:

LINK.EXE /DLL /OUT:MYLIB.DLL FILE1.OBJ FILE2.OBJ FILE3OBJ

When linking your software you may be faced with a situation on which you want to link against several standard shared libraries.
If all the libraries you need exist in a single folder, you can set the LD_LIBRARY_PATH to that folder. By common standard all shared libraries are prefixed with the word lib. If a library exists in LD_LIBRARY_PATH and you want to link against it, you don’t need to pass the entire path to the library, you simply pass -lname and you will link your executable to the symbols of libname.so which should be somewhere inside LD_LIBRARY_PATH.

Tip: You should probably stay away from altering your LD_LIBRARY_PATH, if you do, make sure you keep its original value, and when you’re done restore it, as you might screw the build processes of other software in the system which might depend on what’s on the LD_LIBRARY_PATH.

If you have some other libbar.so library on another folder outside LD_LIBRARY_PATH you can explictly pass the full path to that library /path/to/that/other/library/libbar.so, or you can specify the folder that contains it -L/path/to/that/other/library and then the short hand form -lbar. This latter option makes more sense if the second folder contains several other libraries.

## Useful tools

Sometimes you may be dealing with issues like undefined symbol errors, and you may want to inspect what symbols (functions) are defined in your library.

On Mac there’s otool, on Linux/Mac there’s nm, on Windows there’s depends.exe (a GUI tool that can be used to see both dependencies and the symbol’s tables. Taking a look at the “Entry Point” column will help you understand clearly the difference between symbols linking to a shared library vs symbols linking statically to the same library)

### Useful command options

See shared library dependencies on Mac with otool

otool -L libjlibtorrent.dylib
libjlibtorrent.dylib:
libjlibtorrent.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)


See shared symbols with nm (Linux/Mac)
With nm, you can see the symbol’s name list.
Familiarize yourself with the meaning of the symbol types:
* T (text section symbol)
* U (undefined – useful for those undefined symbol error),
* I (indirect symbol).
*
If the symbol is local (non-external) the symbol type is presented in lowercase letters, for example a lowercase u represents an undefined reference to a private external in another module in the same library.

nm‘s documentation says that if you’re working on Mac and you see that the symbol is preceeded by + or - it means it’s an ObjectiveC method, if you’re familiar with ObjectiveC you will know that + is for class methods and - is for instance methods, but in practice it seems to be a bit more explicit and you will often see objc or OBJC prefixed to those methods.

nm is best used along with grep

Find all Undefined symbols

nm -u libMacOSXUtilsLeopard.jnilib
_CFRelease
_LSSharedFileListCopySnapshot
_LSSharedFileListCreate
_LSSharedFileListInsertItemURL
_LSSharedFileListItemRemove
_LSSharedFileListItemResolve
_OBJC_CLASS_$_NSArray _OBJC_CLASS_$_NSAutoreleasePool
_OBJC_CLASS_$_NSDictionary _OBJC_CLASS_$_NSMutableArray
_OBJC_CLASS_$_NSMutableDictionary _OBJC_CLASS_$_NSString
_OBJC_CLASS_$_NSURL __Block_copy __NSConcreteGlobalBlock __dyld_register_func_for_add_image __objc_empty_cache __objc_empty_vtable _calloc _class_addMethod _class_getInstanceMethod _class_getInstanceSize _class_getInstanceVariable _class_getIvarLayout  ### What to do when lighttpd won’t start and won’t give out any error output? Wednesday, February 4th, 2015 So you upgraded your server, or just all of a sudden you try to start lighttpd, it says the server started ok, but you check and there’s no lighttpd process. You then go after your error log files, and nothing… what the fuck is happening? try this to attempt to debug. sudo strace -ff /usr/local/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf In my case the output showed me lighttpd was having permission issues trying to access a log file… [pid 28073] close(5) = 0 [pid 28073] munmap(0x7fe884c71000, 4096) = 0 [pid 28073] write(3, "2015-02-04 11:04:23: (log.c.164)"..., 49) = 49 [pid 28073] close(2) = 0 [pid 28073] open("/dev/null", O_RDWR) = 2 [pid 28073] brk(0x2203000) = 0x2203000 [pid 28073] open("/home/bh/access.log", O_WRONLY|O_CREAT|O_APPEND, 0644) = -1 EACCES (Permission denied) [pid 28073] write(3, "2015-02-04 11:04:23: (log.c.118)"..., 98) = 98 [pid 28073] write(3, "2015-02-04 11:04:23: (server.c.1"..., 83) = 83 ### 2 Java debugging tricks Tuesday, January 20th, 2015 1. Embed state in your thread names. Some times when you see a crash log with a thread dump, you see the stack of all the threads at the moment of the snapshot, but there’s no way to see what the state of the variables and objects at play in the thread were like. You can use Thread.currentThread.setName(...) and embed variable states on the thread name for a more useful thread dump output. 1. Set a Default uncaught exception handler and log the error to the thread. ### How to resize an EBS (xfs formatted) partition Saturday, December 20th, 2014 First of all, create a snapshot of your EBS volume. Then out of that snapshot you will be able to create your new volume. However, when you detach the old one from your instance and attach the new one, you will still see the old available space with df look at my /dev/xvdf/ available space (after mounting the new EBS volume) ubuntu@ip-10-47-167-74:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       16G  5.9G  9.1G  40% /
udev            7.4G   12K  7.4G   1% /dev
tmpfs           1.5G  176K  1.5G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.4G     0  7.4G   0% /run/shm
/dev/xvdf        20G   19G  1.7G  92% /media/ebs/data  <<< this one


still 20G, I mounted a 80G one!

This drive has been formatted to use an xfs file system. In order to resize it, this is the command I used:

sudo xfs_growfs -d /media/ebs/data


output should be something like this

ubuntu@ip-10-47-167-74:~$sudo xfs_growfs -d /media/ebs/data meta-data=/dev/xvdf isize=256 agcount=4, agsize=1310720 blks = sectsz=512 attr=2 data = bsize=4096 blocks=5242880, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from 5242880 to 20971520  now let’s see the df -h output ubuntu@ip-10-47-167-74:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       16G  5.9G  9.1G  40% /
udev            7.4G   12K  7.4G   1% /dev
tmpfs           1.5G  176K  1.5G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.4G     0  7.4G   0% /run/shm
/dev/xvdf        80G   19G   62G  23% /media/ebs/data


### [bash scripting] How to get a file’s name without its extension(s).

Saturday, September 6th, 2014

Say you have an encrypted file file.foo.gpg and you want to make a shorthand command to decrypt that file, you’ll want the resulting file to be named file.foo (without the .gpg), or say you want the name, with no extension?), you can use bash’s magic variable voodo for that.

A simple version of that script would look something like this:

### GRADLE: How to specify resources from different folders on your sourceSet

Tuesday, July 29th, 2014

Sometimes you need to have resources in your sourceset and these may come from different locations on disk, the official documentation is lacking in real world examples, or at least I just didn’t understand it very well, but from reading forums I finally got it to work.

In the example I specify what files to include/exclude from two different folders.
When the final .jar is created, they’ll keep the package path structure that lives inside the given srcDir folders.

If you just want to add these files (for some reason to the root of your resulting jar, you should make srcDir the full path to where the files live)

sourceSets {
main {
java {
//your java source paths and exclusions go here...
}

resources {
srcDir 'components/resources/src/main/resources'
include '**/*.properties'
include '**/*.png'
include '**/*.gif'
include '**/*.jpg'
include '**/*.html'
include '**/*.js'
include '**/*.sh'
include '**/*.dat'
include '**/*.icc'
exclude '**/*.DS_Store'

srcDir 'common/vuze/azureus2/src'
include '**/Messages*.properties'
exclude '**/*.class'
exclude '**/*.java'
}
}
}


### GRADLE: How to copy files from another .jar into your resulting output .jar

Tuesday, July 29th, 2014

In our project we like to deliver a single jar as the final product, if you need to copy files that live on an existing jar into the Gradle’s output jar, this example shows you how to do that (and more)

jar {
//this is how you change the name of the output jar
archiveName='frostwire.jar'

//some exclusion rules to keep your .jar clean
exclude('META-INF/*.SF', 'META-INF/*.DSA', 'META-INF/*.RSA', 'META-INF/*.MF')

//here we grab all the .class files inside messages.jar and we put them in our resulting jar
from (zipTree('lib/jars/messages.jar')) {
include '**/*.class'
}

//how to manipulate the jar's manifest
manifest {
attributes 'Main-Class': 'com.limegroup.gnutella.gui.Main'
}
}


### GRADLE: How to add a list of local .jar files to the build classpath

Tuesday, July 29th, 2014

Sometimes you don’t want/cant use maven repos, all you have is a bunch of local jars on disk that you want to use as part of your compilation classpath, and the freaking gradle documentation is too vague.

Here is an example:

dependencies {
compile files('lib/jars/gettext-commons.jar',
'lib/jars/lucene-3.5.0.jar',
'lib/jaudiotagger.jar',
'lib/jars/h2-1.3.164.jar',
'lib/jars/messages.jar',
'lib/jars/slf4j-api-1.7.5.jar',
'lib/jars/jaudiotagger.jar',
)
}


### [SOLVED] Gradle: How to increase the Java Compiler’s available Heap Memory

Tuesday, July 29th, 2014

The documentation is not very clear on what all the available options are…
after much Googling and many different attempts finally figured out how to raise the maximum heap of the compiler from within the gradle.build script.

apply plugin: 'java'

compileJava {
//raise heap
options.fork = 'true'
options.forkOptions.with {
memoryMaximumSize = "2048m"
}
}

Update:
So I’ve noticed this works great on MacOSX, but it doesn’t work at all on Windows 8.

The solution to increasing the JVM’s Heap has been to remove those options from gradle.build script and add a new file on the same folder as where your gradle.build file lives called ‘gradle.properties’

These are the contents that made it work for both Mac and Windows (I’ve still not tested this on my linux box)

org.gradle.jvmargs=-Xms256m -Xmx1024m


### [SOLVED] Sublime Text 2: Git binary could not be found in PATH

Tuesday, July 22nd, 2014

Got this annoying dialog popping up on Sublime Text 2?

Go to Preferences > Browse Packages …

a Finder window will open, go to the “Git” folder, open the file called “Git.sublime-settings”

Look for “git_command” and set it’s value to the path of your git executable

(you can find the path of your git executable on the Terminal by typing “which git”)

• ## Categories

• March 2015
• February 2015
• January 2015
• December 2014
• October 2014
• September 2014
• August 2014
• July 2014
• June 2014
• May 2014
• April 2014
• February 2014
• January 2014
• December 2013
• November 2013
• October 2013
• September 2013
• August 2013
• July 2013
• June 2013
• May 2013
• April 2013
• March 2013
• February 2013
• January 2013
• December 2012
• November 2012
• October 2012
• September 2012
• July 2012
• June 2012
• May 2012
• April 2012
• March 2012
• February 2012
• January 2012
• December 2011
• November 2011
• October 2011
• September 2011
• August 2011
• June 2011
• May 2011
• April 2011
• March 2011
• February 2011
• December 2010
• November 2010
• October 2010
• September 2010
• August 2010
• July 2010
• June 2010
• May 2010
• April 2010
• March 2010
• February 2010
• January 2010
• December 2009
• October 2009
• September 2009
• July 2009
• May 2009
• April 2009
• March 2009
• February 2009
• January 2009
• December 2008
• November 2008
• October 2008
• September 2008
• August 2008
• July 2008
• June 2008
• May 2008
• April 2008
• March 2008
• February 2008
• January 2008
• December 2007
• November 2007
• October 2007
• September 2007
• August 2007
• July 2007
• June 2007
• May 2007
• April 2007
• March 2007
• February 2007
• January 2007
• December 2006
• November 2006
• October 2006
• September 2006
• August 2006
• July 2006
• June 2006
• May 2006
• April 2006
• March 2006
• February 2006
• January 2006
• December 2005
• November 2005
• October 2005
• September 2005
• August 2005
• July 2005
• June 2005
• May 2005
• April 2005
• March 2005
• February 2005
• January 2005
• December 2004
• November 2004
• October 2004