Fundamentals of Malware Analysis and Reverse Engineering

Introduction
Most incidents nowadays involve a security breach typically include malware, which might be designed to allow the attacker to remotely control the compromised system, spread within the organization, steal sensitive documents, spy on the victim, and so on. In this blog, we will briefly discuss the key aspects of the malware analysis and reverse engineering investigation process from an incident response point of view, and walks you through an example of using publicly available tools and data sources to start learning about the nature of a suspicious file.

Malware Analysis Techniques
There are two fundamental approaches to analyzing malware, Static and Dynamic.
Static Analysis: Simply examine the malicious program without running it. Sometimes this can help us to confirm whether the file is malware or not. For instance, by viewing the file strings (ASCII and Unicode) we could determine what APIs are being used on a given malware, this could help us build our hypothesis on what the malware is trying to do. On the screenshot below, we can observe that the malicious program is using suspicious APIs, one example is CryptDecrypt, which referencing to Microsoft MSDN "The CryptDecrypt function decrypts data previously encrypted by using the CryptEncrypt function".

This triggered that the program is trying to encrypt a file.

Another example is

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0)
HTTP/1.1
Connection: close

This trigger that the malicious program could be trying to connect to Command-and-Control server after infecting a victim machine.

Lastly
Software\Microsoft\Windows\CurrentVersion\Run

This is a registry key used to cause the program to run each time a user logs in, adversaries often use this registry it to make their malware present on the victim machine.

Dynamic Analysis

Unlike static analysis, Dynamic analysis is performed by observing the behavior of the malware while it is actually running in a live machine, one example is running the application and monitoring the network using packet analyzer such as Wireshark to observe if the malware is trying to connect to the internet. On the screenshot below, we were able to confirm from our observation from the static analysis that the malware is trying to connect to a Command-and-Control server using the same HTTP header we discovered before.

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0)
HTTP/1.1
Connection: close

Usually a debugger is used by the analyst to examine the internal state of a malicious executable. Caution: Before performing dynamic analysis, be sure to do it in a safe environment. Consider deploying a Windows virtual machine to avoid infecting or harm your main host.

Malware Analysis Stages
The picture below demonstrates the different stages of analyzing a malicious program.

The 4 stages are in the form of a pyramid scheme and as we go higher in the pyramid, the complexity of the analysis stage increases. The stages are:

1. Fully-Automated Analysis.
2. Static Properties Analysis.
3. Interactive Behavior Analysis.
4. Manual Code Reversing

Fully-Automated Analysis
Fully automated analysis quickly and simply assesses suspicious files. This method is considered to be the easiest/fastest way to start learning about a particular malware by simply examine it using a fully automated tool such as Cuckoo Sandbox and generate a quick report.

Static Properties Analysis
A good starting point is often to examine static properties of the specimen such as metadata, import tables, hashes, strings, and PE headers. One of the greatest tools used for this type of analyst is PeStudio as shown in the screenshot below.

Interactive Behavior Analysis
After using automated tools and examining the static properties of the file, as well as considering the overall context of the investigation, the analyst might decide to take a closer look at the specimen.
This often involves infecting an isolated laboratory system with the malicious program to observe its behavior as we did before during our Dynamic Analysis. Another example of running the malware in an isolated lab to understand its behavior is by using a tool called Process Monitor.
What we are aiming for from this type of analysis is to have an idea about what the malware is trying to do when it runs on a victim machine.

Manual Code Reversing
Manual code reversing usually involves the use of a debugger, this consider to be an advanced stage of our analysis and requires a good understanding of low-level programming language (Assembly language).
One example of manual code reversing is using the same malware we are analyzed before, as we observed, the malware is using the API CryptDecrypt, this API is used to encrypt data referencing to MDSN as we saw earlier.
Of course, in order to encrypt a file or data you need to have the clear version of it first. The screenshot below shows how manual code reversing could be useful in displaying the content before it has been encrypted.