DPC2021: Beyond Relational: Data storage for modern applications - Mike Lehan

Explore the world of data storage beyond relational databases, discussing key-value stores, document storage, time series databases, and more, and learn how to choose the right database for your application's specific needs.

Key takeaways
  • Key-Value Stores are optimized for fast read operations and do not have relationships between data.
  • They can be used for caching, and Redis is a popular option.
  • Document Storage (e.g., MongoDB) is useful for storing JSON data and allows for efficient querying.
  • Time Series Databases are optimized for storing a large amount of data with a timestamp.
  • Indexes are important for efficient querying in databases.
  • Transactions are important for ensuring data consistency and atomicity.
  • Locks can be used to prevent concurrent updates, but can also cause deadlocks.
  • Queues can be used to decouple producers and consumers, and SQS is a popular option.
  • Webhooks can be used to send notifications to external services, but can be complex to implement.
  • Exactly-once processing is important for ensuring data consistency in distributed systems.
  • Tools like RabbitMQ and Redis can be used to implement message queues and caching.
  • Choosing the right database for the job is important, and there is no one-size-fits-all solution.
  • Considerations such as scalability, data consistency, and query performance should be taken into account when selecting a database.
  • Relational databases may not be the best fit for all problems, and alternative databases like NoSQL databases may be more suitable.
  • The choice of database will depend on the specific requirements of the application and the trade-offs that need to be made.