Talks - Pat Viafore: Extensible Python: Robustness Through Addition

Learn patterns and practices for building extensible Python systems through plugin architectures, data-driven design, and composable interfaces that enable adding features without modifying code.

Key takeaways
  • Make code open to extension but closed for modification - add new functionality without changing existing code

  • Separate policies (business logic) from mechanisms (implementation details) to make systems more flexible and maintainable

  • Use event-driven architectures and message brokers to decouple producers from consumers

  • Leverage data-driven designs by putting policies in declarative formats (YAML, TOML) rather than code

  • Build pluggable architectures using entry points and plugin systems like stevedore

  • Design libraries to be extensible from the start with clear documentation, tests, and composable interfaces

  • Keep dependencies manageable by being mindful of high fan-in/fan-out modules and visualizing dependency graphs

  • Make common operations easy - if simple tasks require many file changes, rethink the design

  • Use dependency injection to make code more flexible and testable

  • Consider the humans - both current and future developers need to understand and maintain the code

  • Break large changes into smaller, focused commits that are easier to review

  • Build composable building blocks that can be combined in different ways rather than monolithic implementations

  • Leverage Python’s ecosystem of plugin systems (pytest, poetry, hypothesis) to extend functionality

  • Store configuration and policies in data formats rather than hardcoding them in source