"Workflows, a new abstraction for distributed systems" by Dominik Tornow (Strange Loop 2022)

Discover how workflows can revolutionize distributed systems, providing a new abstraction that simplifies development, ensures correctness, and guarantees exactly once execution.

Key takeaways
  • Abstractions give compound elements identity and allow for higher-level thinking.
  • A durable execution is an execution that executes exactly once, guaranteeing correctness via completeness, and provides exactly one’s semantics.
  • Volatile executions do not guarantee completeness and may require manual repair in case of failure.
  • A coroutine is a function that yields a closure, an anonymous function, which allows for execution to be suspended and resumed.
  • The runtime invokes a coroutine and stores the results of the task in its log, ensuring that the sequence of tasks is executed correctly even in the presence of failure.
  • The concept of transactions is similar to durable executions, as they guarantee completeness and provide exactly one’s semantics.
  • A proof of concept does not need to be failure-tolerant, but a production system must be.
  • The concept of idempotence refers to the property of an operation that can be safely retried in case of failure without changing the result.
  • The difference between a proof of concept and a production system is that a production system must be failure-tolerant.
  • Abstractions give developers a delightful developer experience, hiding the complexity of failure tolerance and concurrency.
  • The end-to-end argument emphasizes the importance of considering the boundaries of the process itself, rather than just focusing on the individual components.
  • The concept of sameness is important in understanding how durable executions can be composed of lower-level abstractions.
  • The term “abstraction” appears throughout the talk, emphasizing the importance of abstract thinking in software development.
  • The speaker recommends “Data and Reality” by Bill Kent for a philosophical discussion of sameness.