Skip to Main content Skip to Navigation

Formal Approaches for Automatic Deobfuscation and Reverse-engineering of Protected Codes

Abstract : Malware analysis is a growing research field due to the criticity and variety of assets targeted as well as the increasing implied costs. These softwares frequently use evasion tricks aiming at hindering detection and analysis techniques. Among these, obfuscation intent to hide the program behavior. This thesis studies the potential of Dynamic Symbolic Execution (DSE) for reverse-engineering. First, we propose two variants of DSE algorithms adapted and designed to fit on protected codes. The first is a flexible definition of the DSE path predicate computation based on concretization and symbolization. The second is based on the definition of a backward-bounded symbolic execution algorithm. Then, we show how to combine these techniques with static analysis in order to get the best of them. Finally, these algorithms have been implemented in different tools Binsec/se, Pinsec and Idasec interacting alltogether and tested on several malicious codes and commercial packers. Especially, they have been successfully used to circumvent and remove the obfuscation targeted in real-world malwares like X-Tunnel from the famous APT28/Sednit group.
Document type :
Complete list of metadatas

Cited literature [112 references]  Display  Hide  Download
Contributor : Abes Star :  Contact
Submitted on : Wednesday, June 28, 2017 - 12:15:11 PM
Last modification on : Tuesday, April 24, 2018 - 1:34:40 PM
Long-term archiving on: : Wednesday, January 17, 2018 - 9:48:01 PM


Version validated by the jury (STAR)


  • HAL Id : tel-01549003, version 1



Robin David. Formal Approaches for Automatic Deobfuscation and Reverse-engineering of Protected Codes. Cryptography and Security [cs.CR]. Université de Lorraine, 2017. English. ⟨NNT : 2017LORR0013⟩. ⟨tel-01549003⟩



Record views


Files downloads