Designing Config Files: The Conflicting Needs of Programmers and Users — Steven Pool

Use hierarchical structure in config files instead of flat files to reduce redundancy and improve organization

Push configuration capabilities into config files rather than code to give domain experts more control without requiring code changes

Implement runtime overrides to handle variations (like different oven types) without duplicating configs

Add validation and testing:

• Use Pydantic models to validate configs
• Write unit tests to check config differences
• Test which config options are actually used
• Log effective configs during runtime

Keep configs separate from code:

• Store templates (.toml.template) separate from actual configs
• Let domain experts manage their own flat config files
• Avoid embedding logic in code that should be in configs

Document configs properly:

• Export docstrings explaining config effects
• Log actual values used during runtime
• Output effective configs for debugging

Make configs debuggable:

• Include config values in log files
• Create tools to diff configs
• Track which config options are accessed

Structure configs to be:

• Portable and reusable
• Clear and well-organized
• Easy to modify without code changes
• Free of redundant information

Avoid common pitfalls:

• Unused config options after refactoring
• Hidden/implicit dependencies
• Duplicated configuration
• Hardcoded values that should be configurable