How to Benchmark functions in Python - Software Development for Everyone

In this article, we will discuss 4 approaches to benchmark functions in Python

The first 3 methods will help us measure the execution time of a function while the last method will help us measure the memory usage.

Using the time library
Using timeit
Using line_profiler
Using memory_profiler
Conclusion

Using the time library

This is one of the simplest ways to calculate the execution time of a function. We essentially get the time in seconds right before and after we invoke the function. The difference between the two can give us an estimate of the time of the function. I say estimate since this measures the time for a single run. A better benchmark would be running it multiple times and calculating the average time taken.

import time
def func():
   lst = [i for i in range(100000)]
   start = time.perf_counter()
func()
print(f"Completed Execution in {time.perf_counter() - start} seconds")

Below is the output

Completed Execution in 0.007916 seconds

Using timeit

Timeit is a built-in method in Python. It lets us specify the number of times we want to run the function, this helps us to calculate the average time the function takes. This is a better measure than the execution time of a single run. However, for a complex function that takes a long time, this method may not be ideal.

The general syntax is below

timeit.Timer(funcName).timeit(number=number)

number is the number of times we want the function to be executed. It returns the sum of all individual runtimes. To get the average, we can just divide the sum by the number of runs.

Below is how we would measure the average execution time of our function func()

import timeit
num_runs = 10
duration = timeit.Timer(func).timeit(number = num_runs)
avg_duration = duration/num_runs
print(f'On average it took {avg_duration} seconds')

The output is below

On average it took 0.004649160000000001 seconds

We can also use the repeat() method to run the experiment (running the function n times) multiple times. In simpler terms, if we consider an experiment as running our function func() 10 times, we can do this experiment 3 times and get the execution time of each experiment.

The general syntax is

timeit.Timer(funcName).repeat(repeat=num_repetions,number=num_runs)

Below is using it to benchmark our function func()

num_runs = 10
num_repetions = 3
ex_time = timeit.Timer(func).repeat(
                     repeat=num_repetions,
                     number=num_runs)
print(f'It took {ex_time}')

This will return the following output

It took [0.0494772, 0.04936369999999998, 0.048738000000000004]

Similar to the timeit() method, it returns the total time for the 10 runs. We can use the max() to get the worst time, min() to get the best time, and sum(lst)/len(lst) to get the average execution time.

Using line_profiler

line_profiles is a really cool library that gives a line-by-line analysis of a function. Before using the library, we will need to install it

conda install -c anaconda line_profiler

We will need to add a decorator before our function

@profile
def func():
    lst = []
    for i in range(100000):
       lst.append(i)

Since line-profiler provides a line-by-line analysis, it doesn’t make much sense to use it with a list comprehension.

Typ the following commands in the Anaconda Prompt

kernprof -l main.py
python -m line_profiler main.py.lprof

The first command runs our python script and stores the logs in a file. The second command displays the logs in the form of an easy-to-understand table.

Below is the output of the second command

Hit- The number of times that line was executed.
Time- Total time taken by that line
Per Hit- Average time taken by the line
% Time- Fraction of total execution time. As we can see from the above picture, the append function takes around 57% of the execution time.

If you want to change the number of times the function is executed for line_profiler, use the following code

prof = profile(func)
for i in range(10):
   prof()

This will run the function 10 times.

Using Memory_profiler

It is pretty similar to line_profiler, the difference being the logs generated tell us about the memory usage instead of time taken.

First, we will need to install the library

conda install -c anaconda memory_profiler

We will have to add the same decorator as before

@profile
def func():
    lst = []
    for i in range(100000):
       lst.append(i)

Then type the following command in the Anaconda Prompt

python -m memory_profiler main.py

Below is the output

Mem usage– The total memory usage at the line
Increment– memory usage by each execution of that line
Occurrences– number of times the line was executed

Conclusion

If you want a quick time performance test of a piece of code or a function, you should try measuring the execution time using the time library. However, if you want a better estimate, consider using the timeit library.

If you want a more detailed line-by-line analysis, consider using line-profiler and memory-profiler.

Connect with me on LinkedIn

I recently started a modified version of the #100daysofcode challenge. I aim to write content related to Python, Data Science, or Programming every day. Follow my progress on Twitter, Medium, Dev.to, Hashnode, or my WordPress Blog