Endoscope: Unpacking Android Apps with VM-Based Obfuscation

Uncover the secrets of Android app obfuscation with VM-based techniques, learn how to reverse-engineer and unpack malware protected by custom-built virtual machines and custom instructions.

Key takeaways

VM-based obfuscation involves compiling code into bytecode with custom instructions and running it on a custom-built virtual machine, making it difficult to reverse-engineer.
Android VM obfuscation is a growing concern, with many malwares using inexpensive VM-based packers.
To unpack Android apps with VM-based obfuscation, a two-fold methodology is proposed: determine virtualized instructions and handler tables, and then apply mapping relationships to recover bytecode.
The process involves instrumentation, execution tracing, and mapping relationships between virtualized instructions and handlers.
Challenges include identifying handler tables, determining virtualized instructions, and mapping relations between virtualized instructions and handlers.
The proposed solution uses genetic signatures to identify handlers and applies learned mapping relationships to recover bytecode.
Existing unpacking tools are unable to unpack apps protected by VM obfuscation, as the original Delvid bytecode is never placed into memory.
The talk introduces a method to reverse Rhino bytecode of a specific case, and a more general scenario with its challenges and method of unpacking.
The solution involves reconstructing the abstract syntax tree, simulating the interpreter’s stack, and applying learned mapping relationships to recover bytecode.
The talk concludes with an introduction to VM-based obfuscation, its challenges, and a proposed solution for unpacking Android apps with VM-based obfuscation.

Endoscope: Unpacking Android Apps with VM-Based Obfuscation

More talks