Using assert and debug flags to make your Python code faster when it matters

Debugging is an integral part of the software development process, and Python provides several tools to assist with it. Two particularly useful features are the assert statement and the __debug__ built-in constant. These tools primarily allow you to validate assumptions during runtime, ensuring your code behaves as expected. However, they also have a secondary use case: improving the performance of your Python program in a production environment b y removing debugging checks when Python runs in optimized mode.

Why Assertions Matter in Python

Python is a dynamic language, meaning types and many assumptions about your program’s behavior are not enforced or checked at compile time. To catch potential bugs early, you may need to validate these assumptions during execution. The assert statement is a concise tool for this purpose, raising an AssertionError if a condition evaluates to False.

The Role of assert

The assert statement is a simple yet powerful debugging tool. It’s often used during development to ensure that critical assumptions hold true. For instance:

# script.py
import time
 
# Simulate some long-running check
def long_running_check():
    time.sleep(2)
    print("check done")
    return True
 
if __name__ == '__main__':
    print("Hello")
    assert long_running_check() == True  # Validate an important condition
    print("world!")

Here:

1. assert long_running_check() == True ensures that long_running_check() returns True.
2. If the condition fails, the program halts and raises an AssertionError.

However, assertions can introduce runtime overhead, particularly if they involve expensive operations like network calls, database queries, or delays as shown above. For production environments, Python provides a way to eliminate these checks using optimization mode.

Ignoring Assertions with Optimization Mode

Python’s -O flag (optimize mode) removes all assert statements from your code, as well as disables the __debug__ constant. This behavior allows you to write extensive debugging checks during development without affecting the performance of production deployments.

Running in Optimized Mode

$ python script.py
Hello
check done
world!
 
$ python -O script.py
Hello
world!

In optimized mode:

1. The assert statement is ignored entirely, so long_running_check is never called.
2. This improves runtime performance but means that any assumptions checked by assertions are no longer validated.

Using __debug__ for Conditional Debugging

The __debug__ constant is True by default, but it is set to False in optimized mode. This allows you to write conditional debugging logic that is automatically disabled in production.

Example 1: Skipping Expensive Tasks

# script2.py
import time
 
# Simulate some long-running task
def long_running_task():
    time.sleep(2)
    print("task done")
 
if __name__ == '__main__':
    print("Hello")
    if __debug__:
	    long_running_task()
    print("world!")

Running the script with python -O script2.py will skip the long_running_task() function.

Example 2: Combining Conditions with __debug__

# script2.py
import time
 
# Simulate some long-running task
def long_running_task1():
    time.sleep(2)
    print("task 1 done")
 
def long_running_task2():
    time.sleep(2)
    print("task 2 done")
 
if __name__ == '__main__':
    print("Hello")
    x = True
    if __debug__ and x:
	    long_running_task1()
	    
	if x and __debug__:
	    long_running_task2()
	    
    print("world!")

$ python script2.py
Hello
task 1 done
task 2 done
world!
 
$ python -O script2.py
Hello
world!

By strategically combining __debug__ with other conditions, you can finely control debugging behavior in your code.

Example 3: Storing Data during Runtime

When working with large datasets, such as a pandas.DataFrame or data from a REST API request, you may want to store intermediate results for debugging purposes. A common way to use __debug__ here is:

if __debug__:
	df.to_csv('./file_name.csv', sep='\t', encoding='utf-8')

Using __debug__, you can conditionally save this data only during development, without affecting performance in production.

Best Practices for Using assert and __debug__

1) Avoid Side Effects: Assertions and if-branches from __debug__ should not include side effects like modifying the global state of your program. Since assertions are removed in optimized mode, side effects would be skipped and this would likely cause some problems.

2) Use Assertions for Development, Not Production: Assertions are ideal for catching bugs during development but are not a substitute for proper error handling. For runtime checks that must always execute, it is better to use exceptions

# dev/testing:
assert condition, "Critical condition not met"
 
# production:
if not condition:
    raise ValueError("Critical condition not met")

3) Test in Both Modes: Ensure you test your code both with and without optimization enabled to avoid surprises in production.