Christopher Ariza - When and How to Try with Your Own C-Extensions | SciPy 2023

Learn when and how to try C extensions with your Python code to improve performance, including using C types, sliding window approaches, and SIMD, and discover the benefits and opportunities for optimization.

Key takeaways

C extensions can be used to improve performance when working with Python code.
Consider using C types instead of Python objects (e.g., pi objects) when working with arrays.
np.argmax is two orders of magnitude faster than a simple C implementation.
Sliding window approach can be used to find the first true value in a boolean array.
Using pi array operations (e.g., pi array get pointer) can be faster than equivalent Python code.
SIMD can be used to improve performance, especially with modern CPUs.
numpy routines can be slow when dealing with large arrays, and C extensions can be used to improve performance.
Good opportunities for improving performance include using C extensions, avoiding Python objects, and optimizing the inner loop.
npy iter provides a common iteration interface in C and supports everything from iterators to arrays.
simd reduces loop iteration, making the code faster.
numpy has a lot of flexibility, but also a lot of overhead, which can affect performance.
C extensions can be useful for improving performance, especially in cases where the work can be done using C types.
pi array operations can be used to improve performance, especially when working with large arrays.
Iterating over the array in reverse is essentially the same as iterating forward.

Christopher Ariza - When and How to Try with Your Own C-Extensions | SciPy 2023

More talks