Oreo: Protecting ASLR Against Microarchitectural Attacks

Oreo: Protecting ASLR Against Microarchitectural Attacks

Speakers: Shixin Song, Ph.D. Student, MIT

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=CD4-KGJRoIo

Overview

Address Space Layout Randomization (ASLR) is a cornerstone software security mechanism, widely deployed across modern operating systems like Linux, Windows, and macOS. Its primary objective is to randomize the memory locations of key program components, such as executables, libraries, heaps, and stacks, thereby making it significantly harder for attackers to predict the addresses of sensitive code or data. This unpredictability acts as a crucial barrier against common exploitation techniques, particularly code reuse attacks like Return-Oriented Programming (ROP), which rely on knowing the precise memory locations of gadgets.

However, despite its widespread adoption and perceived strength, ASLR has been consistently compromised by a growing wave of microarchitectural side-channel attacks. These attacks exploit subtle, secret-dependent timing or power differences in processor components like caches, Translation Lookaside Buffers (TLBs), and Branch Target Buffers (BTBs) to infer the randomized memory offsets. The talk "Oreo: Protecting ASLR Against Microarchitectural Attacks" by Shixin Song introduces a novel software-hardware co-design mitigation designed to fortify ASLR against these sophisticated bypass techniques.

Oreo tackles the fundamental vulnerability of ASLR by introducing a new architectural layer called mask memory between the virtual and physical memory interfaces. This innovative approach aims to redact ASLR secrets from all microarchitectural structures, effectively eliminating the side channels that attackers currently leverage. By ensuring that microarchitectural behavior remains independent of the ASLR secret, Oreo seeks to restore the intended security guarantees of ASLR, making it resilient against current and future microarchitectural attacks. This work is critical given that kernel ASLR has been deemed "comprehensively compromised" by Google Project Zero, highlighting an urgent need for robust, architectural-level defenses.

Background

▶ Watch: Introduction to Oreo: Protecting ASLR against microarchitectural attacks (0:00)

ASLR's inception aimed to thwart code reuse attacks, where an attacker, having achieved control over the program's instruction pointer, attempts to redirect execution to existing code snippets (gadgets) within the victim program to achieve malicious goals, such as elevating privileges. By randomizing the base address of the program and its modules, ASLR ensures that these gadget addresses are unpredictable to the attacker. Without knowing the secret random offset, any attempt to jump to a hardcoded address would likely land in an invalid memory region, causing a crash and preventing successful exploitation. This randomization acts as a probabilistic defense, forcing attackers to guess the secret offset, which is computationally infeasible for sufficiently large address spaces.

Despite its theoretical strength, ASLR has proven highly susceptible to microarchitectural side-channel attacks. A significant body of research, exemplified by attacks like Flush+Reload, Prime+Probe, and TLB-based attacks, has demonstrated the feasibility of leaking ASLR secrets from user space, and even more critically, from user space to kernel space. The severity of this problem is underscored by findings from Google Project Zero, which documented the practical bypass of kernel ASLR using profusion attacks to corrupt the kernel stack. Their conclusion stated that "kernel ASLR is comprehensively compromised on X6[4] against local attackers and has been for past several years and will be for the indefinite future," emphasizing the critical need for a new defense paradigm.

The core reason ASLR is vulnerable to these attacks stems from two fundamental properties of current processor architectures:

1. ASLR Secret Dependency in Address Translation: During the virtual-to-physical address translation process, the ASLR secret (the randomized bits of the virtual address) is used to index into page tables. This means that different ASLR offsets will cause different page table entries to be accessed. These accesses, in turn, leave secret-dependent traces in microarchitectural structures such as caches, TLBs, and BTBs. For example, an attacker can trigger a system call, causing the processor to access a randomized kernel virtual address. The secret randomized digits (e.g., bits 21) are used to index into the second and third-level page tables. The corresponding page table entry is then cached, creating a secret-dependent side effect in the cache that an attacker can observe via timing attacks.

1. Distinguishable Microarchitectural Side Effects for Valid vs. Invalid Addresses: Attackers can often distinguish between valid and invalid memory addresses based on their microarchitectural behavior. For instance, a common attack gadget involves guessing ASLR secret bits, then probing each guessed address twice. The latency of the second probe is measured. If the first probe hits a valid address, the address translation information is inserted into the TLB. Consequently, the second probe experiences a TLB hit, resulting in significantly lower latency. Conversely, if the address is invalid, no TLB entry is created, and the second probe will take longer due to a full page table walk. Attacks like DRK leverage Intel Transactional Synchronization Extensions (TSX) to perform these probes speculatively, suppressing crashes from invalid accesses while still observing the timing differences. This allows attackers to iteratively deduce the ASLR secret by distinguishing valid from invalid memory regions.

These vulnerabilities highlight that the ASLR secret, intended to be a robust cryptographic-like random value, is inadvertently exposed through the very hardware mechanisms designed for efficient memory management.

Key Findings

▶ Watch: Attack example 1: ASLR secret leakage via page table indexing (3:10)

The central objective of the Oreo project is to make the ASLR secret microarchitecturally independent. This goal is achieved through two primary mechanisms: first, ensuring that ASLR secret bits are not used to index into any microarchitectural structures (like caches, TLBs, or BTBs); and second, making sure that accessing valid and invalid memory addresses results in indistinguishable microarchitectural side effects during transient execution stages.

Oreo introduces a novel, three-layer memory interface by inserting a new layer called mask memory between the traditional virtual and physical memory spaces. The process works as follows:

1. Virtual Memory Division: The virtual memory space is conceptually divided into numerous sub-regions. Due to ASLR, only one of these sub-regions contains the actual, valid code block.

1. Secret Redaction via Masking: Oreo maps all these virtual sub-regions—regardless of whether they contain valid code or not—to the same fixed region within the mask memory. This critical step ensures that all virtual addresses that differ only in their randomized ASLR bits are mapped to an identical mask address. For example, FF21 ABC (a valid randomized address) and FF00 ABC (an invalid address differing only in the ASLR bits) would both map to the same mask address, FF00 ABC. By performing this mapping, Oreo effectively redacts the ASLR secret bits from the virtual address, creating a secret-independent mask address space.

1. Page Table and TLB Integration: The standard page tables are then used to map these secret-independent mask addresses to their corresponding physical memory locations. Crucially, the actual ASLR secret bits are stored in unused bits within the page table entries (PTEs) and within the TLB entries. However, the location of these PTEs and TLB entries in the cache hierarchy remains secret-independent. This means that while the secret is stored for eventual use, it is not used to determine which cache line or TLB entry is accessed, thus preventing secret-dependent indexing side channels.

In summary, the key insight of Oreo is its use of secret-independent mask addresses for all page table lookups and microarchitectural operations during the transient stages of instruction processing. This design directly addresses the first vulnerability of ASLR by ensuring that the secret is never exposed via indexing into microarchitectural structures.

Technical Deep Dive

▶ Watch: Oreo's goal: making ASLR secret microarchitectural independent (5:58)

Oreo's effectiveness hinges on its careful redesign of how ASLR security checks are performed within an out-of-order processor pipeline, particularly in relation to transient execution. In a baseline processor, the validity of a fetched Program Counter (PC) directly influences the behavior of transient stages (e.g., fetch, decode, execute). As previously discussed, probing valid versus invalid addresses results in distinct microarchitectural side effects, such as TLB hits or misses, which can be observed by an attacker.

Oreo fundamentally alters this by ensuring that mask addresses—which are secret-independent—are exclusively used during all transient stages of instruction processing. This means that whether an address would be considered valid or invalid based on the ASLR secret, its representation in the mask memory and its subsequent microarchitectural interactions (cache accesses, TLB probes) are identical. To maintain security, Oreo delays the actual ASLR security check (i.e., verifying if the fetched address matches the correct ASLR offset) until the commit stage of the pipeline.

This delay is critical:

• Indistinguishable Transient Stages: During the fetch, decode, and execute stages, probing what an attacker perceives as valid or invalid addresses (based on their guess of the ASLR secret) will always result in indistinguishable microarchitectural side effects. The mask addresses ensure that cache and TLB behavior is identical, regardless of the underlying ASLR secret. This directly blocks attacks that rely on observing timing differences in these transient stages, such as the profusion attack.
• Crash at Commit: If an attacker speculatively accesses an address that ultimately proves to be invalid (i.e., does not match the true ASLR secret) when the security check is performed at the commit stage, the processor will always cause a crash. This crash prevents the malicious speculative execution from having any architecturally visible effects. While the ASLR secret might be regenerated after a crash, the attacker gains no information about the secret from the microarchitectural behavior leading up to the crash.

A crucial design aspect of Oreo is that this delayed security check does not introduce performance overhead on the pipeline's critical path. The other essential operations, such as fetch and store, are not delayed. The redaction of secret bits into mask addresses is a simple combinatorial logic operation that does not add significant latency.

The Spectre Dilemma and Oreo's Solution

While making valid and invalid addresses indistinguishable in transient stages effectively thwarts ASLR side-channel leakage, it introduces a subtle security dilemma when considering speculative execution attacks like Spectre. If an attacker can speculatively jump to an address with any possible randomized digits, and Oreo makes all these speculative accesses behave identically until commit, then the attacker could potentially execute speculative gadgets without needing to bypass ASLR first. In essence, by making ASLR transparent to speculative execution, Oreo could inadvertently make Spectre-like attacks easier by removing the ASLR barrier for speculative execution.

Oreo addresses this dilemma with a nuanced strategy: **only apply Oreo to protect part of the ASLR randomized bits**. This allows for a trade-off, balancing protection against ASLR bypasses with maintaining resilience against speculative execution attacks.

• Kernel ASLR: In the baseline Linux kernel ASLR, randomization typically occurs for bits below 29. Oreo identifies large, unused regions in the kernel virtual address space that can be randomized with additional bits, extending the randomization up to bit 38. Oreo then applies its protection mechanism only to these extra randomized digits (bits 29-38). This approach ensures that the system's original resistance against Spectre-like attacks (which might target the lower, unprotected bits) is maintained, while significantly strengthening ASLR against side-channel attacks by protecting a larger portion of the address space.

• User Space ASLR: For user space ASLR, many virtual address bits are already randomized by the operating system. Oreo leverages existing non-canonical bits in the virtual address space for its randomization and protection. These non-canonical bits are typically unused in current 64-bit architectures (e.g., bits 48-63 on systems using 48-bit virtual addressing) and can be leveraged to introduce additional, Oreo-protected randomization without interfering with existing ASLR mechanisms or canonical address checks. The Q&A section clarified that the kernel and hardware would be modified to handle these non-canonical addresses, extracting the secret bits during dereferencing, making it compatible with existing position-independent code.

This selective application of Oreo's protection allows the system to be resilient against both ASLR bypasses and Spectre-like attacks, providing a robust defense in a complex threat landscape.

Demo / Proof of Concept

▶ Watch: Oreo's solution: detailed explanation of the mask memory interface (6:30)

The Oreo project includes a practical prototype to validate its architectural design and evaluate its performance and security properties. The prototype involves a software-hardware co-design implementation:

• Software Changes: Modifications were made to the Linux kernel to integrate the new mask memory layer and the revised ASLR security checks.
• Hardware Changes: The hardware components were prototyped and evaluated on Gem5, a cycle-accurate full-system simulator widely used in computer architecture research.

Performance Evaluation

Performance was rigorously evaluated using standard benchmarks, specifically the SPEC CPU2006 and LEBench suites. The key metric measured was cycles per instruction (CPI), comparing performance with and without Oreo enabled. The results demonstrated that Oreo introduces negligible performance overhead across all benchmarks, averaging only 0.11%. This low overhead is a critical finding, supporting the claim that delaying security checks to the commit stage and using combinatorial logic for secret reda does not significantly impact the processor's critical path. The speaker noted that minor observed speed-ups on some benchmarks could be attributed to experimental noise or subtle cache entry differences, rather than actual performance gains.

Security Evaluation

The security efficacy of Oreo was demonstrated using a concrete attack example: the profusion attack.

• Baseline Behavior: In a system without Oreo, the profusion attack successfully distinguishes valid from invalid addresses. Probing a valid address results in shorter latency because its translation information is cached in the TLB. Conversely, probing an invalid address leads to a longer latency due to a TLB miss and a full page table walk. This timing difference allows the attacker to infer ASLR secret bits.
• Oreo's Protection: With Oreo enabled, the security evaluation showed that probing valid and invalid addresses yielded identical latencies. Because Oreo uses secret-independent mask addresses in transient stages and delays the ASLR validity check to the commit stage, the microarchitectural side effects (e.g., TLB hits/misses) become indistinguishable to the attacker. Consequently, the profusion attack, which relies on these timing differences, no longer works against an Oreo-protected system.

These evaluations confirm that Oreo successfully blocks microarchitectural side-channel attacks targeting ASLR while imposing a minimal performance penalty, making it a viable and effective mitigation.

Defensive Implications

▶ Watch: How Oreo maintains ASLR security checks without microarchitectural side effects (8:00)

The development of Oreo represents a significant step forward in securing ASLR against pervasive microarchitectural side-channel attacks. For defenders, the implications of such a software-hardware co-design mitigation are profound:

1. Restored ASLR Integrity: The most immediate implication is the potential to restore the intended security guarantees of ASLR. If widely adopted, architectural solutions like Oreo would render a broad class of existing ASLR bypasses ineffective. This means that attackers would once again face the full probabilistic barrier of ASLR, significantly increasing the cost and complexity of exploiting memory corruption vulnerabilities.

1. Shift in Attack Surface: Oreo fundamentally shifts the attack surface away from microarchitectural side channels related to address translation. Attackers would no longer be able to infer ASLR secrets from cache, TLB, or BTB timing. Instead, any attempts to bypass ASLR would likely revert to older, less efficient techniques like brute-forcing (impractical for large address spaces) or requiring other, non-ASLR-related information leaks.

1. Hardware-Software Co-design Necessity: Oreo underscores the growing recognition that software-only mitigations are often insufficient against hardware-based attacks. For robust security, particularly against microarchitectural vulnerabilities, a coordinated approach involving both hardware and software modifications is essential. Defenders should advocate for and prioritize hardware designs that incorporate such security-by-design principles.

1. Consideration for Speculative Execution: The "Spectre dilemma" addressed by Oreo highlights a critical trade-off that defenders must understand. While protecting ASLR secrets from side channels, care must be taken not to inadvertently weaken defenses against speculative execution attacks. Oreo's solution of selectively protecting ASLR bits (e.g., using unused kernel address space or non-canonical user space bits) offers a pragmatic path forward, suggesting that future architectural mitigations might also require such nuanced approaches.

1. Future-Proofing ASLR: By addressing the root cause of ASLR's microarchitectural vulnerabilities (secret-dependent indexing and distinguishable transient effects), Oreo offers a more future-proof defense against yet-to-be-discovered side-channel attacks that might exploit similar principles. It aims to make ASLR secrets intrinsically independent of observable microarchitectural behavior.

In essence, Oreo provides a blueprint for a more resilient ASLR, pushing the boundaries of memory safety by integrating security directly into the processor's memory management unit and pipeline, thereby offering a more robust defense against the sophisticated attacks of today and tomorrow.

Key Takeaways

• ASLR is a crucial memory safety mechanism widely deployed but critically vulnerable to microarchitectural side-channel attacks.
• Oreo introduces a novel software-hardware co-design mitigation that uses a mask memory layer to redact ASLR secrets from microarchitectural structures.
• The core innovation is ensuring that ASLR secrets are not used for indexing into caches, TLBs, or BTBs, and that valid/invalid addresses have indistinguishable microarchitectural side effects during transient execution.
• Oreo delays ASLR security checks to the commit stage of the processor pipeline, preventing secret leakage while ensuring a crash on invalid accesses.
• To mitigate the Spectre dilemma, Oreo selectively applies its protection to only part of the randomized bits (e.g., unused kernel address space or non-canonical user space bits), balancing ASLR protection with speculative execution resilience.
• The prototype, implemented on Linux with hardware changes on Gem5, demonstrated negligible performance overhead (0.11% average) and successfully blocked the profusion attack.

About the Speaker(s)

Shixin Song is a Ph.D. Student at MIT, advised by Professor M. Ja. His research focuses on computer architecture and security, particularly in developing novel hardware-software co-design solutions to address fundamental vulnerabilities in modern computing systems. The work presented on Oreo exemplifies his contributions to protecting critical security mechanisms like ASLR against advanced microarchitectural threats.