Windows Data Execution Prevention (DEP) from Microsoft is an exploit mitigation technique that forces memory to...
be non-executable unless it is explicitly set as executable by an application. Said more simply, it is a defensive measure for preventing buffer overflows, a technique used by attackers that injects malicious code into the data pages of an application in order to compromise the system when the application reads the data pages. This type of attack is relatively common among malware, which makes the DEP control quite useful. However, DEP does have some limitations.
The first limitation of DEP is that its use is limited to applications compiled to support it. More information on how to use Windows Data Execution Prevention (DEP), as well as other protective features, can be found on the Microsoft website. Another limitation is that the default DEP setting is "Opt In"; applications must be explicitly added to DEP in order to have DEP protection, or the DEP mode must be changed to "Opt Out" or "Always on." (DEP is enabled by default for Internet Explorer 8 on Windows XP SP 3 and newer OSes.) As with anything new that affects the functions of the system, DEP should be tested before being widely used to ensure it does not cause errors in any critical applications.
Despite the limitations of Windows Data Execution Prevention's usability, it is a useful feature and should be considered by anyone who wants a more attack-resistant operating system, particularly when an OS may be missing patches or running applications that could be vulnerable. Internet Explorer (IE) is a prime example -- many organizations still use older versions of IE that contain vulnerabilities, which can result in system compromise. In a January 2010 blog post, Jonathan Ness, head of the MSRC Engineering team of software security engineers at Microsoft, discussed how Windows DEP stopped a new, unknown zero-day attack. Microsoft's recommendation to use DEP to protect against an actively exploited, but unpatched IE flaw is discussed in this short video about the Windows Data Execution Prevention technique.
I recently discussed Windows DEP and other exploit mitigation controls in an article about Windows Enhanced Mitigation Experience Toolkit (EMET). You can use Microsoft's EMET to protect all applications with DEP even if they were not compiled by their respective vendor with DEP support. EMET also contains several other buffer overflow-focused protections and should be considered (and tested!) by anyone considering adopting the use of DEP. The additional protections of EMET also help by adding and/or improving memory organization and validation features in Windows.
In summary, DEP is a helpful countermeasure that will increase your level of security, but DEP alone will not make you 100% secure. As the saying goes: Security is a journey, not a destination.
About the author:
Tom Chmielarski is a senior consultant with GlassHouse Technologies, Inc.