Building an ML Factory | Adi Hirschtein, Iguazio (May 2022)

Automation

Building an ML factory requires efficient orchestration, automation, and monitoring. Discover the key components, including feature store, model monitoring, and automation, and how a serverless architecture can enhance collaboration and reduce costs.

Key takeaways

Effective machine learning (ML) development involves a combination of orchestration, automation, and monitoring.
Building an ML factory requires addressing challenges such as scalability, resource management, and feedback loops.
Key components include:
- Feature store for efficient and scalable feature engineering
- Model monitoring for drift detection and accuracy maintenance
- Automation for streamlined development, deployment, and maintenance
Serverless architecture provides benefits such as flexible resource allocation, cost-effectiveness, and enhanced collaboration
Companies like S&P Global and Samsung are using this approach to achieve real-time recommendation and fraud prediction respectively
Data scientists can focus on their core tasks while the platform handles infrastructure and management
The platform provides a single pane of glass for monitoring features, models, and resources, enabling data scientists to work more efficiently
Use cases include:
- Online and real-time recommendation systems
- Fraud prediction and detection
- Image recognition and extraction
- Graph-based modeling and serving
Challenges include managing complexity, handling cold starts, and ensuring scalability and cost-effectiveness
Addressing these challenges requires a combination of technical expertise, domain knowledge, and cultural shift

Building an ML Factory | Adi Hirschtein, Iguazio (May 2022)

More talks