Playing Video Games One Frame at a Time - Ólafur Waage - NDC TechTown 2023

Discover how to play video games one frame at a time using image recognition, pattern matching, and advanced programming techniques, including Frida library and OpenCV, to automate gameplay and improve performance.

Key takeaways
  • Playing games programmatically with image recognition and pattern matching
  • Using template matching in ImageTV for image recognition
  • Creating a system to play Bejeweled 3 automatically
  • Using Frida library for controlling video games remotely
  • Hooking game functions to play game sequences automatically
  • Using OpenCV for color detection and image recognition
  • Implementing Coyote Time for smoother gameplay updates
  • Running games in debug mode for testing and debugging
  • Using detours library to intercept game functions
  • Writing approval tests for game automation
  • Using async/await pattern for game state updates
  • Writing unit tests for game functions
  • Replacing game functions with custom functions for automation
  • Using image manipulation and pattern recognition for gameplay control
  • Implementing random number generation for game sequences
  • Creating custom game states for automation
  • Using OpenCV for color detection
  • Using Frida library for controlling video games remotely