We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
PyData Chicago: Running Notebooks in Production? Blessing or Curse? by Eduardo Blancas
"Explore the benefits and challenges of using Jupyter notebooks in production. Discover best practices for maintaining and testing notebooks, managing versioning, and scheduling notebooks. By Eduardo Blancas at PyData Chicago."
- Jupyter is a file format (JSON) and a platform with features allowing editing of notebooks.
- Maintaining long notebooks can be challenging, but keeping the code clear and concise with a standard structure helps maintainability.
- Having immutable code that doesn’t change arguments is a way to avoid potential issues with notebooks in production.
- Following a common standard for formatting, such as Black, can improve readability and maintainability.
- Testing notebooks can be challenging, but libraries such as testbook can extract and test functions from notebooks.
- Data pipelines can help break down data analysis logic into manageable pieces and test them in isolation.
- External libraries used in notebooks should be listed and recorded to prevent versioning issues.
- Data snapshots can help keep track of the expected outputs of notebooks.
- Formatting notebooks and using alternative formats, such as .py files with annotations, can help improve the maintainability and interoperability of notebooks in production.
- Scheduling notebooks can be done using tools such as Papermill or a scheduler that supports bash commands.