How we achieved extreme scale under the hood in Azure Functions - Paul Yuknewicz - NDC Oslo 2024

Azure Functions Flex Consumption plan enables extreme scale with ability to burst to 1000+ instances in under a minute and handle millions of requests per second

Concurrency control is a key mechanism for optimizing scale and cost efficiency - allows controlling how many messages/requests each instance handles in parallel

Cold start improvements achieved through placeholder pools - pre-warmed VMs/containers ready to be specialized with application code

Scale decisions now made through direct monitoring of queues and traffic patterns rather than unreliable ping-based monitoring

Target 70% CPU/memory utilization for optimal hardware efficiency while maintaining performance headroom

Run from package deployment recommended for faster cold starts and better scaling

Rightsizing instance sizes (smaller instances with scale out vs larger instances) provides better cost efficiency

Monitor queue depth and throughput as key metrics for scaling behavior

Logging levels impact scaling performance - use minimal logging in steady state

Flex Consumption runs natively on Linux with VNet support and private networking capabilities