Exploit Mitigations

Definition

Exploit mitigations are the layered defensive technologies — compile-time, runtime, OS-level, and hardware-level — that interrupt the chain from memory-corruption bug to working exploit. Individually, each mitigation defeats one primitive (stack canaries detect return-address overwrite; ASLR randomizes address layout; DEP makes data pages non-executable; CFI restricts indirect-branch targets; MTE catches use-after-free). Collectively, they force attackers to combine multiple primitives — typically an info leak plus a control-flow primitive plus a chain technique like ROP — to land an exploit, raising exploitation cost by orders of magnitude. This note is the defender-pair to memory-corruption and stack-buffer-overflow; every exploitation technique in the branch is shaped by which mitigations are present.

Why it matters

The history of binary defense is the history of exploit mitigations:

• 1996 — Aleph One publishes "Smashing the Stack for Fun and Profit". Stack overflows yield trivial code execution.
• 1998–2001 — StackGuard / ProPolice / -fstack-protector introduces stack canaries.
• 2003 — OpenBSD ships W^X (writable XOR executable). Windows XP SP2 adds NX bit support; Linux follows.
• 2005 — ASLR enters mainstream OS distributions.
• 2006 — Mark Dowd / Ben Hawkes publish ROP, demonstrating that DEP can be bypassed without code injection.
• 2009 — Vista/7 enable ASLR + DEP by default. Modern exploitation begins requiring info-leak primitives.
• 2014 — Microsoft introduces Control Flow Guard (CFG). Clang ships CFI shortly after.
• 2020 — Intel CET (shadow stack + ENDBR for indirect branches) ships in Tiger Lake CPUs.
• 2021 — ARM Memory Tagging Extension (MTE) ships in Pixel 8 / iPhone 15+.
• 2023+ — Industry-wide consensus: the only structural solution is memory-safe languages (Rust, Swift, modern C++).

A modern exploit against a hardened target is almost never a single bug. It is a chain: info leak + memory-corruption primitive + control-flow hijack + payload, each defeating one or two mitigations. Senior practitioners think in terms of which mitigations the target has and which primitives are needed to defeat each. Without this mental model, you cannot reason about whether a CVE is exploitable in practice.

The class also matters as a teaching note because it surfaces three transferable senior facts:

• Mitigations are not "anti-virus". They do not detect malicious behavior; they detect specific corruption primitives at specific points (canary mismatch on ret, indirect branch to non-ENDBR target, MTE tag mismatch on access). The detection is mechanical and architectural — much harder to bypass than signature-based AV.
• Every mitigation has a defined bypass class. ASLR is defeated by info leaks. DEP is defeated by ROP/JOP. Stack canaries are defeated by info leaks (read the canary) or by primitives that don't touch the canary. CFI is defeated by data-only attacks and signature-matching gadgets. Senior usage names the bypass class, not just the mitigation.
• The order of arrival matters operationally. Canaries (1998) + DEP (2003) + ASLR (2005) + CFI (2014) + Shadow Stack (2020) form a cumulative cost-stack on attackers. An attacker facing all of them simultaneously needs primitives that defeat each. This is why modern Pwn2Own exploit chains are 4–8 bugs deep.

How it works

Exploit mitigations split into 6 mechanism families, each defeating a specific exploitation primitive:

1. Stack-frame protection. Stack canaries / SSP. A random per-process value placed between local variables and the saved return address. Checked on function exit. Corruption that crosses it triggers __stack_chk_fail. Shadow stacks (Intel CET, ARM PAC). A second, write-protected copy of the return address. The CPU compares against it on ret; mismatch traps. Hardware-enforced; cannot be bypassed by canary-style info leaks. Defeats: classic stack buffer overflow → return-address control. Bypass class: canary-leak primitives, or corruption that doesn't cross the canary (in-frame locals, frame-pointer overwrite).

2. Code-data separation (DEP / NX / W^X). No memory page can be both writable and executable. Data pages (stack, heap, BSS) are marked non-executable; code pages are marked read-only. Defeats: injected shellcode on the stack or heap. Bypass class: ROP and JOP — reuse existing executable code instead of injecting new code.

3. Address Space Layout Randomization (ASLR / PIE / KASLR). Randomizes the base address of executable, libraries, stack, heap, and (in PIE binaries) the binary itself at load time. The attacker doesn't know where gadgets, libc functions, or buffers live in memory. Defeats: hardcoded gadget addresses, hardcoded shellcode addresses. Bypass class: info-leak primitives that disclose a single address from each randomized region; once leaked, all addresses in that region are computable.

4. Control Flow Integrity (CFI). The compiler tags every indirect-call target as "valid" (Intel ENDBR/ENDBRANCH instructions, Clang CFI tables, MSVC CFG bitmap). On indirect branch, the CPU or runtime verifies the target is a valid landing pad. Invalid targets trap. Defeats: most ROP/JOP chains, vtable hijacking, function-pointer overwrites. Bypass class: data-only attacks (corrupt non-control data to achieve the same effect), signature-matching gadgets (CFI permits any legal function entry; attacker picks one with useful side effects), call-oriented programming.

5. Memory tagging (ARM MTE, HWASAN). Each 16-byte memory granule carries a 4-bit "color" tag. Pointers carry the expected color in their high bits. CPU traps on color mismatch at load/store. Catches use-after-free, heap overflow, out-of-bounds at hardware speed. Defeats: the majority of heap-based memory corruption. Bypass class: tag-aliasing collisions (1-in-16 odds per attempt — viable for some primitives), explicit tag manipulation if the attacker has read/write primitives on tagged regions.

6. Sandboxing and process isolation. Untrusted code runs in a restricted process: seccomp filters (Linux), App Sandbox (macOS/iOS), Win32k filters (Windows), JIT sandbox isolation, browser renderer sandbox. Exploited code in the sandbox can do limited damage; reaching the privileged side requires a second exploitable bug. Defeats: RCE in a renderer/parser from producing immediate system compromise. Bypass class: sandbox-escape bugs (themselves usually kernel memory corruption or IPC validation bugs). Most browser-RCE Pwn2Own chains include 2+ bugs solely for sandbox escape.

A representative "with vs without mitigations" comparison for the same stack buffer overflow:

WITHOUT mitigations (1996-style):
  bug primitive: overwrite saved RIP
  exploit: write shellcode in buf, point RIP at &buf
  cost: 30 lines of Python, 1 hour

WITH modern stack mitigations (2026 default):
  bug primitive: overwrite saved RIP
  blocked by: stack canary, ASLR, DEP, CET shadow stack
  exploit requires:
    - info-leak primitive to read canary (separate bug or format-string)
    - info-leak primitive to read libc base (defeat ASLR)
    - ROP chain (defeat DEP)
    - CET-aware control-flow target (CET-permitted indirect-branch target or stack-pivot to ROP)
  cost: 200+ lines of pwntools, weeks of research, often a second bug

The bug is not "mitigations are bypassable"; it is each mitigation defeats a specific primitive, and the cumulative stack imposes a cost-multiplier that determines whether a CVE is practically exploitable.

Techniques / patterns

• Always enable the full modern mitigation set at compile time. GCC/Clang: -fstack-protector-strong -D_FORTIFY_SOURCE=3 -fPIE -pie -fcf-protection=full -Wl,-z,relro,-z,now -fstack-clash-protection. MSVC: /GS /guard:cf /guard:ehcont /CETCOMPAT /DYNAMICBASE /NXCOMPAT /HIGHENTROPYVA. The marginal performance cost is < 1% on most workloads; the exploit-cost increase is orders of magnitude.
• Verify mitigation status with checksec or readelf. The checksec tool (part of pwntools or as a standalone script) reports which mitigations are active in a given binary. Production binaries should report Canary=Yes, NX=Yes, PIE=Yes, RELRO=Full, RUNPATH=No.
• Audit dependencies, not just first-party code. A statically-linked library compiled without -fstack-protector provides no canary on functions it owns. Modern hardened distributions (Ubuntu Pro hardening, Red Hat Enterprise Linux) build their package set with full mitigations enabled; rolling your own toolchain forfeits this.
• Hardware mitigations require both compiler + kernel + CPU. Intel CET needs: CPU support (Tiger Lake+), kernel support (Linux 6.6+ with CET enabled), and the binary compiled with -fcf-protection=full. ARM MTE needs: ARMv8.5+ CPU, kernel support, malloc implementation that tags allocations, binary linked against the tagging-aware libc. Senior usage verifies the full stack.
• Mitigations interact with each other. Stack canaries protect the saved return address but not adjacent stack locals; CFI protects indirect branches but not direct calls; ASLR is defeated by a single info leak that breaks the whole randomization. Reasoning about which combination protects against which primitive is the senior skill.
• The cost-multiplier varies by target. Embedded / IoT firmware often ships with zero mitigations; consumer Linux with strong defaults; Chrome / Edge with PartitionAlloc + V8 sandbox + MiraclePtr on top of the OS-level defaults. The same memory-corruption bug class has wildly different exploitation cost depending on where the bug lives.
• OPSEC: mitigation-aware exploits produce mitigation-aware telemetry. EDR products detect canary failures, CET violations, unusual VirtualProtect calls, and ROP-gadget patterns. The mitigation-bypass primitives often leave high-fidelity signals even when they succeed.

Variants and bypasses

The mitigation landscape splits into 5 deployment layers, each with its own bypass surface.

1. Compile-time mitigations

Stack canaries, FORTIFY_SOURCE, RELRO/BIND_NOW, PIE, CET-compatibility flags. Set at build time; cost is paid by the compiler. Cannot be retroactively added to binaries — re-compilation is required. Most-common deployment gap: third-party closed-source binaries compiled without modern flags.

2. OS-level runtime mitigations

ASLR (kernel-managed randomization), DEP (kernel page-table enforcement), KPTI (kernel page-table isolation against Meltdown), SMEP/SMAP (supervisor-mode execute/access prevention). Configured by the OS at process startup. Bypassable by info leaks (ASLR) or by composite primitives (KASLR + KPTI together).

3. Hardware mitigations

Intel CET (shadow stack + indirect-branch tracking), ARM PAC (pointer authentication), ARM MTE (memory tagging), ARM BTI (branch-target identification). Enforced by the CPU; cannot be bypassed by software alone. Require both new hardware and updated OS/toolchain support.

4. Allocator hardening

Chrome's PartitionAlloc, Microsoft Segment Heap, mimalloc-secure, MiraclePtr (reference-counted dangling-pointer protection), GuardedMalloc. Sit between userspace code and the kernel; trap common heap-corruption primitives. The browser-grade defense layer; less common in general server software.

5. Language-level memory safety

Rust, Swift, Go, modern Java, C#, TypeScript. The structural fix — the bug class becomes impossible to express in the language. Cost: rewriting C/C++ code. Microsoft Azure, Linux kernel (rust-for-linux), Android (over 50% of new code in 2023+) are moving aggressively in this direction.

Impact

Ordered by typical real-world severity (of defensive misconfiguration — the technique is defensive):

• A single missing mitigation can collapse the cost stack. A high-value binary compiled without PIE leaks gadget addresses for free; without canary, single-overflow exploits become trivial; without RELRO, GOT-overwrite primitives work. The exploit cost is set by the weakest mitigation in the stack.
• Third-party closed-source dependencies often lag. Vendor binaries from 2018-era SDKs may ship without modern flags. Bundling these into your application imports the vulnerability surface they would have at that compile-time.
• Embedded / IoT firmware routinely ships with no memory mitigations. The cost stack is the 1996 baseline. This is why router/camera/printer compromise remains industry-wide trivial.
• Mitigation-bypass research drives exploit-tool development. Every named exploitation framework (Metasploit, Cobalt Strike, Sliver) ships mitigation-aware payload generators. The "exploit one bug" world hasn't existed for over a decade on modern targets.
• Hardware-only mitigations have rollout latency. ARM MTE requires ARMv8.5+; Intel CET requires Tiger Lake+. Fleet-wide deployment is multi-year. Older hardware never gets the protection.

Detection and defense

This note is a defense — it does not have a target-side defender section in the usual sense. The relevant framings:

Verification — confirm mitigations are actually applied

1. Audit production binaries with checksec. checksec --file=/usr/bin/myservice reports each mitigation's status. Production servers should be audit-able at any time; surprise findings (a binary compiled without canary in 2024) drive ticketing.

2. CI gate on checksec output. For first-party builds, fail the CI pipeline if any of: Canary=No, NX=No, PIE=No, RELRO=Partial. Treat mitigation regression as a build break.

3. Verify hardware features are enabled. Linux: dmesg | grep -i "smep\|smap\|cet\|mte". macOS: sysctl hw.optional. Android: getprop ro.arm64.memtag.*. Production fleets should track hardware-feature deployment.

4. Audit dependency mitigations. checksec on every shared library and statically-linked dependency. Vendor SDKs and prebuilt libraries are the recurring weak link.

Detection of exploitation attempts

When mitigations fire, they produce telemetry:

5. Canary failures. __stack_chk_fail aborts produce process-crash events with a recognizable signature. EDR and SIEM rules on stack-canary aborts are high-fidelity exploit indicators.

6. CET violations. Intel CET shadow-stack mismatches and indirect-branch-target violations produce hardware exceptions delivered to the OS as SIGSEGV with specific signal codes. Detection rules on these events catch many ROP/JOP attempts.

7. MTE tag mismatches. ARM MTE generates SIGSEGV with SEGV_MTEAERR (asynchronous) or SEGV_MTESERR (synchronous). On Android Pixel devices, Google's crash-reporting pipeline aggregates these as high-priority memory-safety signals.

8. EDR runtime detection of ROP patterns. Behavioral signatures on unusual VirtualProtect/mprotect calls, ROP gadget chains in stack memory, JIT-spray patterns. Layered on top of the structural mitigations.

What does not work as a primary defense

• "We have antivirus." AV is signature-based and does not detect memory-corruption primitives. Mitigations operate at the architectural layer, where signatures don't apply.
• "Just enable ASLR." ASLR alone is defeated by a single info leak. Layered mitigation is the structural answer; ASLR is one layer.
• Trusting "default mitigations are on" without verification. Default flags vary by distro, by compiler version, by build system. checksec is cheap; verify.
• Disabling mitigations "for performance". The cost is usually < 1%; the security cost of disabling is orders of magnitude. Profile first; the suspicion is almost always wrong.
• Assuming binary mitigations cover scripting languages. Python, JavaScript, Ruby, PHP run in interpreters/JITs. The mitigations of the interpreter protect interpreter memory; bugs in interpreted code itself (logic errors, injection) are a different category.

Practical labs

# Lab 1 — Install and run checksec on system binaries.
# checksec ships with pwntools, or as the standalone script:
#   curl -O https://raw.githubusercontent.com/slimm609/checksec.sh/master/checksec
#   chmod +x checksec; ./checksec --file=$(which sudo)
checksec --file=$(which sudo)
checksec --file=$(which ssh)
checksec --file=$(which bash)
# Expected: Canary=Yes, NX=Yes, PIE=Yes, RELRO=Full on modern Linux distros.
# Any No is a finding.

# Lab 2 — Compare the same vulnerable program with progressive mitigations.
cat > vuln.c <<'EOF'
#include <stdio.h>
#include <string.h>
int main(int argc, char **argv) {
    char buf[64];
    if (argc > 1) strcpy(buf, argv[1]);
    printf("Got: %s\n", buf);
    return 0;
}
EOF

# Build with NO mitigations:
gcc -fno-stack-protector -no-pie -z execstack -o v0 vuln.c
# Build with canary only:
gcc -fstack-protector-strong -no-pie -z execstack -o v1 vuln.c
# Build with canary + PIE:
gcc -fstack-protector-strong -fPIE -pie -z execstack -o v2 vuln.c
# Build with canary + PIE + NX + RELRO:
gcc -fstack-protector-strong -fPIE -pie -Wl,-z,relro,-z,now -o v3 vuln.c
# Build with full modern mitigations:
gcc -fstack-protector-strong -D_FORTIFY_SOURCE=3 -fPIE -pie -Wl,-z,relro,-z,now \
    -fcf-protection=full -fstack-clash-protection -O2 -o v4 vuln.c

for bin in v0 v1 v2 v3 v4; do
  echo "=== $bin ==="
  checksec --file=./$bin
done
# Observe progressive mitigation status.

# Lab 3 — Demonstrate that the same exploit progressively fails.
# Run each binary with a long overflow input:
for bin in v0 v1 v2 v3 v4; do
  echo "=== $bin ==="
  ./$bin $(python3 -c 'print("A"*200)') 2>&1 | head -3
done
# Expected:
#   v0: silent corruption or shellcode-territory crash
#   v1-v4: '*** stack smashing detected ***: terminated' (canary catches it)
# The bug is the same; the failure mode is controlled.

# Lab 4 — Verify Intel CET is active on the running system.
dmesg 2>&1 | grep -i "x86/cet"
cat /proc/cpuinfo | grep -o 'cet_ss\|cet_ibt' | sort -u
# Expected on Tiger Lake+: cet_ss and/or cet_ibt entries.
# If absent, CET-aware compilation produces ENDBR instructions but no enforcement.

# Lab 5 — Inspect ENDBR instructions in a CET-compiled binary.
gcc -fcf-protection=full -O2 -c -o vuln_cet.o vuln.c
objdump -d vuln_cet.o | grep -E "endbr32|endbr64" | head -10
# Expected: ENDBR64 instructions at the start of every indirect-branch-targetable function.
# These are the landing pads CET enforcement requires.

# Lab 6 — Audit a third-party binary you didn't compile.
# Pick a vendor-provided binary in /opt, /usr/local, or a Docker image:
for f in /opt/*/bin/* /usr/local/bin/* ; do
  [ -x "$f" ] && [ ! -L "$f" ] && checksec --file="$f" 2>/dev/null
done | grep -E "(No\b|Partial)" | head -10
# Every result is a hardening gap. The vendor's binary inherits whatever toolchain they used.

Practical examples

• Sudo CVE-2021-3156 ("Baron Samedit"). Heap-adjacent off-by-one stack-style overflow. Despite stack canaries, ASLR, and PIE — all enabled on Sudo by default — the bug yields local root on essentially every distribution. The takeaway: mitigations raise cost, do not guarantee invulnerability. The chain still went through.
• PartitionAlloc + MiraclePtr collapses a Chrome UAF. In 2023, Chrome's MiraclePtr (reference-counted dangling-pointer protection for some heap regions) downgraded ~50% of historically-exploitable UAFs to "abort instead of exploit". Browsers became measurably harder to pwn at the renderer layer. The mitigation didn't fix the bugs; it broke the exploitation chain.
• Print Spooler CVE-2021-1675 / PrintNightmare. A Windows kernel-mode bug exploitable despite SMEP, SMAP, KASLR, KPTI, and CFG — because the attacker controlled a kernel-mode code path that bypassed CFG via DLL loading (a documented Windows kernel feature). Architecturally-permitted operations can defeat architectural mitigations.
• ARM MTE on Pixel 8 catches a real-world UAF. Google's pre-launch testing of Pixel 8 with MTE-enabled malloc surfaced multiple UAF bugs in shipping Android apps that had not been detected by ASAN/HWASAN coverage. The hardware mitigation caught what software fuzzing had missed.
• Embedded device with zero mitigations. A common cheap IP camera, firmware audit reveals: no canary, no PIE, no NX, no ASLR. A trivial CGI buffer-overflow in the web management interface yields root with a 50-line exploit script. The same bug class on a modern Linux server with full mitigations would require a chain of 4+ primitives.

Related notes

• memory-corruption — the bug class this note's mitigations target.
• stack-buffer-overflow — the canonical worked example whose mitigation landscape is described here.
• rop-and-ret2libc — the technique that defeats DEP/NX and remains the modern exploitation foundation.
• Attacker-Defender Duality — exploit mitigations are the cleanest expression of the duality in the entire vault — every offensive technique produced a corresponding mitigation, and every mitigation produced a corresponding bypass class.
• CIA triad — mitigations protect program integrity; their failure modes preserve availability (controlled abort) over confidentiality/integrity (silent corruption).
• EDR / process correlation — when mitigations fire (canary failure, CET violation), EDR captures the signal.
• Behavioral vs Signature Detection — mitigation-bypass attempts produce behavioral signals (unusual control-flow, anomalous VirtualProtect) rather than signature-detectable code patterns.
• Windows Privilege Escalation — modern kernel mitigations (SMEP, SMAP, CFG, kCET) are the kernel-side counterparts of the user-space mitigations in this note.

Suggested future atomic notes

• stack-canaries-and-shadow-stacks — SSP / Intel CET shadow stack / ARM PAC mechanical deep dive.
• aslr-pie-and-info-leak-chains — the bypass class for ASLR; the most-common defeat primitive in modern exploitation.
• control-flow-integrity-cfi — Intel IBT / Clang CFI / MSVC CFG mechanics.
• arm-mte-and-memory-tagging — hardware memory tagging deep dive.
• seccomp-and-sandbox-isolation — Linux process-sandbox mechanism.
• partitionalloc-and-miracleptr — Chrome's allocator hardening as a case study.
• meltdown-spectre-and-side-channel-mitigations — the speculative-execution mitigation family (KPTI, IBRS, RETPOLINE).

References

• Foundational: Microsoft Security Response Center — A proactive approach to more secure code — https://msrc.microsoft.com/blog/2019/07/a-proactive-approach-to-more-secure-code/
• Research / Deep Dive: Dennis Andriesse — Practical Binary Analysis (No Starch Press, 2018) — chapter on modern exploit mitigations
• Hardening: Intel — Control-flow Enforcement Technology (CET) specification — https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf
• Hardening: ARM — Memory Tagging Extension (MTE) — https://developer.arm.com/documentation/108035/latest/
• Hardening: Linux kernel — Self-Protection documentation — https://www.kernel.org/doc/html/latest/security/self-protection.html
• Research / Deep Dive: Project Zero — The State of the Art in Bypassing Memory Safety Mitigations — https://googleprojectzero.blogspot.com/