Theodore Meynard - MLflow workshop | PyData London 2023

Automation

Machine learning model deployment and real-time iteration with MLflow and Airflow, overcoming challenges in accuracy, latency, and infrastructure scalability, and building a robust system with error detection and customization.

Key takeaways

The speaker faced challenges in deploying a machine learning model to production, including handling user signals, data sets, and model changes.
The team used MLflow for model training and deployment, and Airflow for orchestration and automation.
The speaker emphasized the importance of having a clear interface (e.g., MLflow) and being able to analyze and iterate on the model in real-time.
The team faced issues with model accuracy and latency, and had to implement solutions such as incremental model updates and health checks.
The speaker discussed the importance of having a scalable and fault-tolerant infrastructure (e.g., Kubernetes) to handle model deployment and inference.
The team used sampling data to validate and test the model, and then deployed it to production using MLflow and Airflow.
The speaker highlighted the challenges of handling model changes and updates in production, and emphasized the importance of having a system in place to detect and handle errors.
The team used Argo for deployment and rollback of the model, and Airflow for orchestration and automation.
The speaker mentioned the importance of having a clear understanding of the user signals and data sets used in the model, and the need for customization and personalization.
The team used Databricks for data processing and analysis, and MLflow for model training and deployment.
The speaker discussed the importance of having a strong infrastructure and automation in place to handle model deployment and inference, and highlighted the challenges of doing so.
The team used an open-source platform (e.g., MLflow) and Kubernetes for scalability and fault tolerance.
The speaker emphasized the importance of having a clear understanding of the model’s performance and limitations.
The team used incremental model updates and health checks to ensure the model’s performance and accuracy.
The speaker discussed the importance of having a system in place to detect and handle errors, and highlighted the challenges of handling model changes and updates in production.

Theodore Meynard - MLflow workshop | PyData London 2023

More talks