Talks - Juan Altmayer Pizzorno: Near Zero-Overhead Python Code Coverage

Coverage overhead has been a significant problem, with traditional tools adding up to 200% overhead using Coverage.py

Slipcover achieves near zero-overhead coverage by:

• Using bytecode instrumentation instead of sys.trace
• Implementing probes as native extension modules
• Only recording first hits rather than every execution
• Batching instrumentation operations

Python 3.12 introduced sys.monitoring API which:

• Allows more selective enabling of coverage
• Provides better performance than sys.trace
• Makes implementing coverage tools simpler
• Reduces overhead to around 9% for line coverage

Branch coverage is more complex than line coverage:

• Shows paths taken/not taken in code execution
• Requires mapping bytecode jumps to source code
• Currently has higher overhead than line coverage
• Still needs work to fully implement with sys.monitoring

Key performance improvements achieved:

• Reduced overhead from 200% to about 5% for line coverage
• Branch coverage reduced to around 31% overhead
• Can now potentially run coverage in production
• Enables faster fuzzing and property-based testing

Coverage tools are valuable for:

• Finding dead code
• Guiding fuzzing operations
• Verifying test completeness
• Property-based testing (22x faster with Slipcover)