Azure Messaging, Queueing, and Events - Which service is right for your solution? - Brian Gorman

Learn the key differences between Azure messaging services like Event Hub, Event Grid, Service Bus & Storage Queues to choose the right one for your architecture needs.

Key takeaways
  • Azure provides several messaging and event services, each optimized for different scenarios:

    • Event Hub - For big data streaming, IoT telemetry, millions of events per second
    • Event Grid - For reactive programming, event monitoring and responding to state changes
    • Service Bus - For enterprise messaging, ordered delivery, transactions
    • Storage Queues - For simple queuing needs with large storage requirements (>80GB)
  • Key distinctions between services:

    • Messages vs Events - Messages contain raw data and state that needs processing, Events are notifications something happened
    • At-most-once vs At-least-once delivery
    • Ordering guarantees (Service Bus can guarantee order with sessions)
    • Storage capacity (Storage Queues >80GB, Service Bus <80GB)
    • Throughput requirements (Event Hub handles millions/sec)
  • Event Hub specifics:

    • Uses partitions for scale and ordered processing
    • Consumer groups enable parallel processing
    • Capture feature for cold storage path
    • Throughput Units control ingress/egress capacity
  • Service Bus features:

    • Topics and subscriptions for pub/sub
    • Message sessions for ordered processing
    • Dead letter queues
    • Message TTL and scheduling
    • RBAC and SAS token authentication
  • Best practices:

    • Use AMQP instead of HTTPS for better performance
    • Consider message size limits (64KB for Storage Queues)
    • Enable auto-capture for Event Hubs when needed
    • Use managed identities over connection strings
    • Implement proper error handling and dead letter queues
  • Integration capabilities:

    • Logic Apps for event-driven workflows
    • Stream Analytics for real-time analytics
    • Functions for serverless processing
    • IoT Hub compatibility with Event Hub SDK