Compiling Software

Installing, Configuring, and Building Software in Linux

If you're too impatient (read: lazy) to read through all this, you can also watch the Scientific Computing Summer Program presentation here: Software Presentation.

Source vs. Binary

Most software available for Linux and Unix is open-source, and is provided for free. This software is often provided in two forms: a source package, and a binary package.

Source packages consist of the source code, makefiles, scripts, and other things required to compile the source code. Because you're compiling the source code, on YOUR machine, with YOUR compiler, it doesn't matter what platform you're running. When you compile the code, you will have an executable file that will run on your computer.

Binaries are pre-compiled executables. A Linux binary is different from a Mac OS X binary, which is different from a Windows binary. Often, there may be different binaries even for a single platform (for example, a Mac OS X 64-bit binary is different from a Mac 32-bit binary, and the 64-bit binary will not run on a 32-bit machine). The binary is pre-compiled, which means the developer has already compiled the source code for that particular platform.

Downloading and running a binary package for your platform is simple to do - you simply download the binary and run it from the command line. To run a Linux or Unix binary, you would type:

$ /path/to/program/bin/program

(Note that the binary will be located in the "bin" folder 99% of the time). To run a Mac binary, you would move the .app file into your Applications folder (or wheverever you would like to run the binary from). And in Windows, you would double-click the .exe file.

Downloading The Source Package

Source files are available on a software project's website. You'll likely see several different versions, so you'll want to get the latest version. Versions are released when problems with the software are fixed, so the latest version will probably have more features and fewer problems. However, if you see a version that is labeled "alpha" or "beta", that means it has not been completely tested yet, and so it's not stable and may crash or behave strangely.

The packages are usually in a compressed format, like .tar.gz or .tar.bz. These can be unzipped using:

$ tar xvzf program-version.tar.gz

This creates a folder called 'program-version'. This folder contains all the source code that will be compiled in order to run the program.

When you move into the folder that was created and list the files in the directory, you'll probably see several files with names like README or INSTALL. It is a good idea to read these, as the instructions given here will not work for buil ding every program. The README and INSTALL files will give you some basic instructions on how to build and configure the program.

Configuring the Program for Installation

The first step in building your program is to configure it. This is a way of "customizing" the program to fit your needs. Often, a program's default installation settings are not enough to fit your needs. For example, if you have a 64-bit processor, the program will not install a 64-bit version by default (because most people don't have a 64-bit processor). You can specify that you have a 64-bit processor when you configure the program. You can also control whether features are enabled or disabled, where the program installs itself, and where it looks for various programs it might depend on.

To configure your program, you'll execute a file called "configure", which is essentially a script that checks to make sure you have everything you need. You run the script with various flags (meaning you add --flag to the end, where "flag" is replaced by some special keyworkd that tells the program how to behave). The configure script will check to ensure you have all the prerequisite software, all the libraries you need, that your hardware matches the configure options you gave it (that is, it makes sure that you don't try to install a 64-bit version on a 32-bit machine), and lots of other things.

Since the options available for "configure" vary greatly from program to program, you need a way of seeing what options you have available to you. To do this, you will run configure with a special flag - the help flag.

$ ./configure --help

The ./ at the beginning tells Unix to run the command named "configure" that is located in the current folder, instead of looking for a program already installed on your computer named configure.

This will print out a lot of output that will give you a long list of different flags and what each will do. For example, here is the output from running this command for the program Subversion:

75%

Most of these options are very specialized and the average user will not need them. In fact, if you don't care about customizing the installation, you could run ./configure without options. When you don't specify a value for a flag, the configure script will use a default value.

Sometimes, however, you will need to configure your program with specific options, so the values for various flags will need to be set. Sometimes it is only one or two flags, but sometimes you may need to change as many as 10 flags, sometimes more, and set special environmental variables or compiler flags to make your compiler work like normal. It can be a hassle, if something goes wrong in the configure script, to re-type the whole line, or to keep editing the configure line on the command line. You may also lose the configure line you used, which can be frustrating if something happens to the program and you need to re-install it, or if you need to install it on another system and you forgot that one special flag you need to make everything work.

In this case, you can make a configure script. You can call it anything you'd like, but as an example I will use runconfigure.sh. The script will be a very simple script that, when run, will simply run your configure line. You can make the file using your favorite text editor, and it will look like this:

#!/bin/sh

./configure \
--flag-1=THIS \
--flag-2=THAT \
--flag-3=THE_OTHER \
VARIABLE=value

The first line tells the system that this is a script, and the script should be run using the program /bin/sh. The next lines run configure with the flags flag-1 set to the value THIS, flag-2 set to THAT, and flag-3 set to THE_OTHER. The backslash \ at the end of each line makes sure that everything is put together as one single command (if there were no backslash, it would run the first line as a command, then run the second line as a separate command, and so on).

One last step remains before you can run your custom configure script - you have to make it executable. You can do this using chmod, which changes the permissions of a file:

$ chmod u+x runconfigure.sh

Your custom configure script can be run just like the normal configure script:

$ ./runconfigure.sh

This will give the same output as running ./configure except that it will run configure with the options you want.

Making Your Program

Once you've configured your program, you have told the program what you want it to install and where. The next step is to compile all of the source code into the executable files. This involves using the GNU make program. The make program uses a file called a Makefile, which is essentially a special script containing detailed instructions about how to compile each file that's part of a project. Each file needs to link to the files it refers to, and include files it uses, and the Makefile ensures each file is linked or included in the instructions given to the compiler.

Running configure sets certain variables used by GNU make. Once you have run configure, you can run GNU make by typing:

$ make

Or, in some cases,

$ make all

You can always see what is available to install by using help:

$ make help

This will display a list of different packages that can be made (also called "make targets"). Running "make" makes the default targets, and running "make all" makes all targets.

Installing Your Program

In some cases, typing make will only compile each source code file in-place - meaning, if you specify that you want to the program to install to /path/to/installation, and you are running configure and make in /path/to/source/code, then when you type make, and when once make finishes, the finished product will reside in /path/to/source/code, and not in /path/to/installation. In these cases, you must issue a second command:

$ make install