Talks - David Hewitt: How Python Harnesses Rust through PyO3

PyO3 is a Rust project that provides bindings to Python, allowing seamless integration between Python and Rust code through standard import statements

Rust adoption in Python packages is growing rapidly, with downloads doubling yearly - projections suggest Rust usage could match C/C++ in the Python ecosystem within 2-4 years

PyO3 aims to preserve Python’s developer experience while adding Rust’s power and precision - developers can write Rust code that behaves like native Python modules

Rust provides key benefits for Python projects:

• Memory safety and thread safety guarantees
• Protection against data races through the type system
• Performance improvements (2-4x) with minimal code changes
• Enhanced security that appeals to major tech companies

PyO3 handles important safety considerations:

• Managing Python reference counting
• Protecting against panics crossing language boundaries
• Coordinating with Python’s Global Interpreter Lock
• Converting between Python and Rust types automatically

The integration works through:

• Generated code that bridges Python and Rust
• Function trampolines that maintain safety invariants
• Python’s import system for loading Rust modules
• Matching Python’s calling conventions

Key tools for getting started:

• Maturin for project setup and building
• Setup Tools Rust as an alternative build system
• Optional Rust-Numpy integration for matrix operations
• Development guides and documentation

PyO3 is particularly valuable for:

• Security-critical Python code
• CPU-intensive operations needing performance
• Multi-threaded Python applications
• Projects requiring memory safety guarantees

The project aims for low barriers to entry, allowing Python developers to gradually adopt Rust where beneficial while maintaining Python’s ease of use

Future developments include better support for Python 3.12 sub-interpreters and Python 3.13’s experimental free threading mode