Last week I kicked off a series of blogs with a discussion of how an effective IPS solution can fight obfuscation techniques by malware. This week, we’ll look at how JavaScript poses a danger when combined with PDF files.
One of the easiest and most powerful ways to customize PDF files is by using JavaScript. JavaScript in PDF files implements objects, methods, and properties that enable one to manipulate PDFs, produce database-driven PDFs, modify the appearance of PDFs, and much more. However, one effective method for evading detection systems is to use malicious JavaScript code.
Malicious JavaScript in PDFs is usually encoded in different ways to bypass detection by intrusion prevention system (IPS) engines. The highlighted portion in the packet capture below shows the garbled values that represent the injected malicious JavaScript.
What kind of detection system can spot these attacks?
There are various forms of obfuscation: Encoding can be coupled with concatenation, calculation techniques (the transformation of bytes or values), and hidden shell code in irrelevant buffers. These techniques make it hard to detect any malicious traffic in a data packet. Generally if there is no obfuscation involved, the IPS engine can perform static analysis on the traffic and find malicious patterns. If the payload is encoded, however, the static-analysis engine cannot detect the presence of malicious JavaScript because pattern matching becomes inefficient. Thus the IPS engine must perform deep parsing to decode packets and run static analysis that looks for patterns. It is also crucial for the detection engine to execute the complete JavaScript code to obtain the decoded original payload and clearly determine any malicious activity. Any security solution without these capabilities will be subject to evasion by obfuscation.
McAfee’s Network Security Platform (NSP) can decode the encoded traffic and raise an alert if the encountered pattern seems malicious.
How McAfee’s NSP detects encoded malicious JavaScript
When traffic passes through McAfee’s IPS engine, the encoded JavaScript is automatically decoded and executed by the NSP, which checks for any known malicious traffic. If NSP encounters malicious content, it alerts the administrators and stands ready to perform any action they choose. Our encoded JavaScript sample is decoded by NSP as shown below:
Decoding encoded traffic and analyzing it for malicious JavaScript is an essential feature of IPS products. McAfee’s NSP 7.x does this. More enhancements to improve detection of encoded malicious JavaScript will be added to NSP 7.5–watch for updates on this blog.
Special thanks to Chong Xu, Director of IPS Research, for his assistance with this post.