Data Execution Prevention (DEP) is a memory protection feature available in modern operating systems, Linux, Mac OS X, iOS, Microsoft Windows and Android. DEP allows memory pages to be designated as non-executable. Its duty is to prevent the content of a region of memory to be executed as instructions by a program, service, device driver, etc. With DEP enabled, pages belonging to stack, data segments and heep are protected against exploits. An application receives an exception with the status code STATUS_ACCESS_VIOLATION if it tries to execute code from a page that is marked as non-executable.
DEP runs in two modes: hardware-enforced DEP for CPUs that can mark memory pages as nonexecutable, and software-enforced DEP with limited protection for CPUs that do not have hardware support. Software-enforced DEP does not protect against execution of code in data pages, but counters SEH overwrite, another type of attack.
DEP was introduced on Linux in 2004 (kernel 2.6.8), on Windows in 2004 with Windows XP Service Pack 2, while Apple introduced DEP when they moved to x86 in 2006.
Hardware-enforced DEP enables the NX bit on compatible CPUs, through the automatic use of PAE kernel in 32-bit Windows and the native support on 64-bit kernels. Windows Vista DEP works by marking certain parts of memory as being intended to hold only data, which the NX or XD bit enabled processor then understands as non-executable. This helps prevent buffer overflow attacks from succeeding. In Windows, from version Vista, whether DEP is enabled or disabled for a particular process can be viewed on the Processes tab in the Windows Task Manager.
Software DEP (with no NX bit) is what Microsoft calls their enforcement of “Safe Structured Exception Handling”. Software DEP/SafeSEH simply checks when an exception is raised to make sure that the exception is registered in a function table for the application, and requires the program to be built with it. However, even though it creates an impression that software DEP is related to the prevention of executing code in data pages, it is a different form of protection.
In Microsoft Windows if an x86 processor supports DEP, and the system’s BIOS supports it and it has been enabled, either by the manufacturer or the user, then the NX features are turned on in Windows, on a limited “OptIn” basis. (PAE must also be enabled in 32-bit Windows.) This setting provides protection only for a limited set of Windows system and binary files. To achieve full protection, the user must choose either “OptOut” (“turn on DEP for all program services except those I select”), covering all programs and processes not specifically exempted, or “AlwaysOn”, covering all. These are configurable through the System Properties interface. If DEP is not supported by the particular x86 processor used, no protection is given. Intel’s IA-64 architecture also has a form of NX, supported by Windows.
OptIn: DEP is enabled for system modules only (user application must explicitly opt in)
OptOut: DEP is enabled for all modules (user application must explicitly opt out)
AlwaysOn: enables DEP for all applications
AlwaysOff: disables DEP for all applications
How to view current system policy
Open command line, change directory to %systemroot%\system32 and execute:
bcdedit.exe /enum all | findstr "nx"
How to set DEP policy
bcdedit.exe /set nx AlwaysOn
How to examine DEP status for an application
1. Download Process Explorer
2. Run it
3. Edit displayed columns to add DEP status column. (Menu->View->Select Columns..)