We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
Sarah Gibson No Magic Added Deploying Multiple JupyterHubs to Multiple Clouds from one Repositor
Deploying multiple JupyterHubs to multiple clouds from a single repository, simplifying hub management and reducing engineer dependencies, with a focus on replication, automation, and naming conventions.
- Deploy multiple JupyterHubs to multiple clouds from a single repository
- Goal: maintain right to replicate and ease hub management
-
Solve:
- One file per cluster, confusing config
- Overcommitted engineers, long-running pipeline
- Magic (dependence on someone else’s servers) needed for deployment
-
Solution:
- Separate config into individual files per hub
- Use Deployer convenience function to automate deployments
- Introduce cluster and hub naming convention
- Log to remember which projects each hub is run on
-
Benefits:
- Easier to identify which config files relate to which hub
- Easier for communities to extract and manage their infrastructure
- Reduced dependency on magic (outsourcing to someone else’s servers)
-
Challenges:
- Preserve right to replicate (maintain identical infrastructure)
- Handle multiple clouds and clusters
- Avoid single point of failure (engineer dependency)
-
CI/CD:
- Create JSON objects to describe production and staging hubs
- Run staging job, filter out failed staging, run production
- Upgrade helm charts for each cluster
-
Future developments:
- Post markdown tables as comments on pull requests
- Implement dynamic parallel matrix strategies