Documenting R&D Progress using jupyter-book - and feel safe for the next performance audit

Learn how to document R&D projects with Jupyter Book, combining research notes, code, and progress tracking for seamless performance audits and reproducible results.

Key takeaways
  • Jupyter Book combines daily work environment, documentation, and R&D progress tracking in one solution, making it ideal for performance audits

  • Key components include:

    • Well-organized file system structure
    • Automated workflow for documentation
    • Integration with Python ecosystem
    • Support for Markdown and Jupyter notebooks
    • Non-destructive processing workflow
  • Documentation structure follows:

    • Root directory for each component
    • Separate sections for simulations, tests, and studies
    • Changelog and README files
    • Directory structure for photos, videos, and supplementary materials
  • Evaluation process:

    • Reference notebooks that can be reused
    • Standardized data processing workflows
    • Automated rebuilding of documentation when new tests/studies are added
    • Support for interactive data visualization (bokeh)
  • Technical implementation details:

    • Uses Sphinx under the hood
    • Simple pip installation process
    • Config.yaml file defines book structure
    • Table of contents auto-generated from directory structure
    • Extension support for additional features (videos, custom formatting)
  • Best practices:

    • Use isolated Python environments
    • Keep raw data separate from processing
    • Implement version control compatible workflow
    • Maintain clear changelog documentation
    • Automate documentation rebuilding process
  • Raw data management:

    • JSON-encoded pandas DataFrames
    • Separate storage for measurement data
    • Reproducible processing pipelines
    • Support for various data formats and visualizations