Petsc

Petsc is the Portable Extensible Toolkit for Scientific Computation. It's used for the parallel solution of PDEs and provides parallel linear and nonlinear solvers. Petsc is written in C.

Installation

Petsc 2.3.3

Petsc 2.3. can be installed using the following configure line. This will require installation of MPI (I suggest OpenMPI).

/path/to/source/of/petsc-2.3.3/config/configure.py \
   --prefix=/path/to/petsc-2.3.3 \
   --with-matlab=false \
   --with-x=false \
   --with-shared=0 \
   --with-mpi-dir=/path/to/mpi \
   PETSC_DIR=/path/to/source/of/petsc-2.3.3

To make this a debug build, add --with-debugging=1 to the configure line.

The build process consists of:

1. run the configure wrapper
2. turn on lam with lamboot (or whatever MPI you're using)
3. make all
4. make install
5. make test (this script should pass all tests OK)

Petsc 3.0.0

Petsc 3.0.0 can be installed using the configure line:

/path/to/source/of/petsc-3.0.0/config/configure.py \
   --prefix=/path/to/petsc-3.0.0 \
   --with-matlab=false \
   --with-x=false \
   --with-shared=0 \
   --with-mpi-dir=/path/to/mpi \
   --with-clanguage=c++ \
   --with-fc=0 \
   PETSC_DIR=/path/to/source/of/petsc-3.0.0

The build process is similar to the above:

1. run the configure wrapper
2. turn on lam with lamboot (or whatever MPI you're using)
3. make all
4. make install
5. make test (this script should pass all tests OK)

Flags explanation

The --with-clanguage=c++ flag is required if you are building Petsc to be used from a C++ program. Otherwise, Petsc will try and use a C compiler, which will cause a lot of problems with C++ code.

The --with-fc=0 flag turns off Fortran. I don't use the Fortran interface, and unfortunately Fortran is the cause of many problems on OS X (thank you Apple for your lack of standards). I leave it out so that I won't have to deal with additional errors related to things I will never use.

Building with Petsc

Source code

There are a number of different header files that must be included in any source code that uses Petsc objects or functions. These must be included by using:

#include "headerfile.h"

Depending on the functionality used, different header files must be used.

Header file	Function
petsc.h	generic header file, should be included unless another header file would be more applicable
petscvec.h	vector structure and functions
petscmat.h	matrix structure and functions
petscksp.h	functions solving linear systems using Krylov subspace methods
petscpc.h	preconditioner structure and functions

To determine, more specifically, which header file should be included, visit the Petsc documentation page (http://www.mcs.anl.gov/petsc/petsc-as/documentation/index.html), and locate the documentation page for the function or object of interest. The page will indicate which of the header files listed must be included.

A compiler directive can be used in C or C++ code to check if Petsc is being used:

#ifdef HAVE_PETSC 
#include "petsc.h"
#endif

This is useful if, for example, you want to build a program that will run with or without Petsc.

Petsc Makefiles

Main article: Example Petsc Makefile

Note: this section will presume you are building a C++ program. If you're building a C program, there are many examples provided by Petsc. These can be found using:

$ cd /path/to/src/of/petsc-3.0.0/src

$ find . -name examples -type d

./contrib/semiLagrange/examples
./dm/adda/examples
./dm/ao/examples
./dm/da/examples
./dm/mesh/examples
./ksp/ksp/examples
./ksp/pc/examples
./ksp/pc/impls/is/feti/examples
./mat/examples
./snes/examples
./sys/draw/examples
./sys/error/examples
./sys/examples
./sys/random/examples
./sys/viewer/examples
./tops/examples
./ts/examples
./vec/is/examples
./vec/pf/examples
./vec/vec/examples

For more on building Makefiles, see Make.

Compilation of a C or C++ program using Petsc is best done with Makefiles, since there is a large amount of information that needs to be fed to the compiler, and Petsc provides that information.

To begin, information contained in variables that make can understand must be included using make's include command:

include ${PETSC_DIR}/conf/base

in turn, ${PETSC_DIR}/conf/base includes several other files. The information contained in these files includes linker and compiler commands, -L and -l library flags for compilers, include directories, etc.

A Makefile for a C++ program utilizing Petsc may be constructed using the following structure:

Section 1: Variable declaration

As shown above, several variables must be declared. The path to the Petsc directory is useful, so it is put into the $(PETSC_DIR) make macro. Additionally, other variables are defined by including files provided by Petsc.

PETSC_DIR=/path/to/petsc-3.0.0

include ${PETSC_DIR}/conf/base

Section 2: Files

There are lots of ways to keep your Makefiles short and to the point. First, a list of files containing source code can be defined:

SRC_FILES = \
    BoundaryConditionFactory.cc \
    BoundaryCondition.cc \
    Field.cc \
    FileIO.cc \
    JacobiSolver.cc \
    Timer.cc \
    TimerFactory.cc \
    GmresSolver.cc \

and a make macro can be used to create an analogous list, but replacing all ".cc" suffixes with ".o":

OBJ_FILES = $(SRC_FILES:%.cc=%.o)

Section 3: Make targets

There are a couple of useful make targets one can define.

The first target should define how to build non-Petsc object files:

%.o: %.cc
    g++ -c -Wall -I. $< -o $@

The second target should define how to build Petsc object files. This example encapsulates all Petsc-related calls in the driver and in a GMRES solver class, so the target is only defined for those file. If there were more files using Petsc, a list could be constructed, and a make target generated from that rule.

GmresSolver.o: GmresSolver.cc
    ${PETSC_COMPILE} -c -Wall -I. -I$(PETSC_DIR)/inc

This rule uses some include directives to point the compiler to the Petsc header files and objects used in the source code file GmresSolver.cc.

Finally, the driver make target, which compiles the driver and links all compiled objects to the driver, requires Petsc libraries to be linked in as well (see the Presentations page, Scientific Computing Summer Workshop 2, for details).

Laplace: Laplace.cc $(OBJ_FILES)
    -${CLINKER} Laplace.cc -o bin.x $(OBJ_FILES) ${PETSC_LIB} -I$(PETSC_DIR)/include

A couple of things to note here. First, when linking my driver source code, I'm adding ${PETSC_LIB} (which adds all of the library links that I will need) because I've got an #include "petsc.h" and #include "petscksp.h" in my driver source code (the ${PETSC_LIB} variable includes all Petsc libraries during linking, so it's good to use it instead of, say, ${PETSC_KSP_LIB}, just to be safe). Second, I have to link everything using -${CLINKER} (minus sign and all). If I were to compile Petsc to use C, then $CLINKER would point to mpicc, or whatever MPI C-compiler my MPI distribution happened to provide. However, using the configure flag --with-clanguage=c++ makes the variable ${CLINKER} point to mpic++, or whatever C++-compiler my MPI distribution happens to provide.

See Also

• C++
• Example Petsc Makefile
• Hypre
• Make

• Computational fluid dynamics (lecture notes)
• Partial differential equations (lecture notes)