Python/Profiling

Introduction

List of Python Profiling Tools

Built Into Python

Profiling tools provided by Python:

• cProfile is a C extension, the recommended method of profiling python programs, and is based on lsprof
• profile is a module that adds significant overhead; reports time spent in calls to built-in functions and methods
• hotshot - not developed/supported anymore

Benefits:

• Easy
• Built-in

Downsides:

• As stated by rkern [1], "The current profiling tools supported in Python 2.7 and later only time function calls. This is a good first step for locating hotspots in one's program and is frequently all one needs to do to optimize the program. However, sometimes the cause of the hotspot is actually a single line in the function, and that line may not be obvious from just reading the source code."

Link: https://docs.python.org/2/library/profile.html

Line by Line

line_profiler module:

• Can time code by adding decorators
• Dumps a binary file that gives a line by line breakdown of time spent within a given function

To install:

$ pip install line_profiler

To profile:

@profile
def slow_function(a, b, c):
    ...

This results in a file called script_to_profile.py.lprof, which you can read with the line_prof module:

$ python -m line_profiler script_to_profile.py.lprof

Link to github: https://github.com/rkern/line_profiler

Graphviz Dot

Gprof2dot:

• Reads all sorts of input formats, outputs to graphviz dot

Gprof2dot link: https://github.com/jrfonseca/gprof2dot

cProfile

Basic usage

The basic way to use cProfile is to create a profile object, and to enable and disable profiling using that object. The default behavior is to print a summary when it is finished.

import cProfile

def factorial(n):
  if(n==1):
    return 1
  else:
    return n*factorial(n-1)

pr = cProfile.Profile()
pr.enable()
factorial(800)
pr.disable()