Distributed Hash Tables, Video, and Fun! - Thomas Gebert & Nick Misturak

Discover the world of Distributed Hash Tables, Video Sharing, and Fun! Learn about DHT algorithms, routing, distance metrics, security, and more, as well as the speaker's humorous take on JavaScript, Clojure Script, blockchain, TCP vs UDP, and HLS.

Key takeaways
  • Distributed Hash Tables (DHTs) vs. Redis and Memcache: DHTs are decentralized and do not rely on a central authority, whereas Redis and Memcache are centralized.
  • Cadimlia: A popular DHT algorithm used by Facebook, Blizzard, and others, which creates a routing table to facilitate efficient lookup of nodes.
  • Routing: Each node maintains its own routing table, divided into buckets, and uses hole punching to solve the problem of initiating a connection between two nodes.
  • Distance Metric: The XOR of two IDs is used as a distance metric, which allows for efficient lookup and traversal of the address space.
  • Security: Current implementation of Cadimlia is not inherently secure and may be vulnerable to attacks such as Sybil attacks.
  • Fault Tolerance: Cadimlia attempts to maintain a constant connection to nodes with open ports to ensure fault tolerance.
  • Distributed Video Sharing Platform: The talk presents a design for a distributed video sharing platform that uses Cadimlia to store and retrieve videos.
  • JavaScript vs. Clojure Script: The speaker finds that JavaScript and Clojure Script share similarities in their approaches to functional programming and immutable data structures.
  • Reagent: A library that provides a way to generate HTML-like elements in Clojure Script, which simplifies the process of creating user interfaces.
  • Blockchain: The speaker jokingly mentions the idea of using blockchain to incentivize people to donate their bandwidth and CPU cycles to the distributed video sharing platform.
  • TCP vs. UDP: The speaker discusses the trade-offs between using TCP and UDP for network communication, with UDP being more commonly used in DHTs.
  • HLS (HTTP Live Streaming): A web standard that allows for efficient streaming of video content, which can be used in conjunction with Cadimlia.