Serverless Deployments with Canary – Creating DevOps engineers at LEGO.com | Nicole Yip

Learn how LEGO.com's DevOps engineers use serverless deployments with Canary to achieve continuous integration and deployment, monorepo structure, automated pipelines, and a DevOps mindset.

Key takeaways
  1. Continuous Integration and Deployment:

    • The team used CircleCI for continuous integration and deployment.
    • Pull requests triggered unit and integration tests, and successful builds deployed to QA.
    • Canary deployments with 10% traffic shifting were used to gradually roll out new versions.
  2. Monorepo Structure:

    • The team used a monorepo structure with separate directories for each service.
    • This allowed for efficient testing and deployment of individual services.
  3. Automated Deployment Pipelines:

    • Deployment pipelines were automated using CircleCI workflows.
    • Pipelines were triggered by commits to the develop branch and manual release approvals.
  4. Visibility and Context:

    • A custom Mission Control dashboard provided visibility into the deployment status of all services.
    • Developers could easily see which services were being deployed and to which environments.
  5. Unintuitive Release Triggers:

    • The team initially used get tags to trigger deployments, which was confusing and error-prone.
    • They later switched to using the CircleCI API to trigger deployments, which provided more control and flexibility.
  6. Ability to Recover Quickly:

    • Canary deployments allowed the team to quickly roll back changes if they caused problems.
    • Alerts and monitoring were used to detect issues and notify on-call engineers.
  7. Adding New Services:

    • A selective script was used to only deploy the service that was changed in a commit.
    • This made it easy to add new services to the deployment pipeline.
  8. DevOps Mindset and Culture:

    • The team focused on creating a DevOps culture where developers were responsible for the entire lifecycle of their services.
    • This included development, testing, deployment, and monitoring.
  9. Tooling and Automation:

    • The team used a variety of tools and automation to streamline their deployment process.
    • This included CircleCI, AWS CodeDeploy, and custom scripts.
  10. Continuous Improvement:

    • The team continuously monitored and improved their deployment process.
    • They identified pain points, implemented solutions, and measured the results.