ERW-Radar: An Adaptive Detection System against Evasive Ransomware by Contextual Behavior Detection and Fine-grained Content Analysis

ERW-Radar: An Adaptive Detection System against Evasive Ransomware by Contextual Behavior Detection and Fine-grained Content Analysis

Speakers: Lingbo Zhao

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=-h4vl94eq0U

Overview

This article delves into ERW-Radar, an innovative adaptive detection system designed to combat the growing threat of evasive ransomware. Presented by Lingbo Zhao at the NDSS Symposium, the talk addresses critical shortcomings in traditional ransomware detection mechanisms that are increasingly bypassed by sophisticated attack techniques. Unlike conventional ransomware that exhibits obvious, high-intensity I/O behaviors, evasive variants deliberately modify their operational patterns to fly under the radar, making them exceptionally challenging to identify using existing security solutions.

ERW-Radar introduces a multi-faceted approach, combining contextual behavior detection with fine-grained content analysis to identify ransomware that sacrifices encryption efficiency to weaken or hide malicious features. This includes ransomware that adjusts I/O strategies to reduce frequency, imitates benign programs, or employs partial encryption and data padding to lower perceived entropy. The system specifically targets ransomware that encrypts the majority of file content, acknowledging that extremely slow encryption rates (e.g., one byte per hour) or scenarios involving privileged attackers shutting down detection systems are outside its primary scope. The research highlights the critical need for adaptive and intelligent defense systems capable of discerning subtle, long-term malicious patterns amidst seemingly benign activity.

The significance of ERW-Radar lies in its ability to adapt to the dynamic tactics of modern ransomware. By moving beyond static signatures and simplistic behavioral thresholds, it offers a robust defense against threats that are specifically engineered to evade detection. The system's design principles emphasize balancing detection accuracy with computational efficiency, ensuring that its protective mechanisms can be deployed without imposing undue overhead on system performance, a crucial factor for real-world applicability.

Background

▶ Watch: Introduction to evasive ransomware and its types (0:00)

Traditional ransomware typically operates by encrypting a large number of files in a short timeframe, leading to highly anomalous I/O behaviors that are relatively easy for established security systems to detect. These systems often rely on predefined features or specific patterns associated with rapid, high-volume file modification. However, the ransomware landscape has evolved significantly, with attackers developing sophisticated techniques to bypass these defenses. This new breed, termed evasive ransomware, deliberately sacrifices encryption efficiency to weaken or conceal their malicious footprints.

The talk identifies three primary categories of evasive ransomware techniques that challenge existing detection paradigms:

1. Adjusted I/O Strategy: Attackers may modify their I/O operations to be less intensive or more regular, significantly reducing the I/O frequency during encryption. This allows the ransomware to operate below the detection thresholds of systems that are tuned for high-volume, rapid changes. Current approaches, heavily reliant on rigid feature sets, are easily bypassed by such adaptive I/O throttling.
2. Imitation of Benign Programs: Some evasive ransomware attempts to mimic the I/O patterns of legitimate applications. Existing detection systems often assume a clear behavioral distinction between ransomware (due to frequent encryption and erasure) and benign programs, making them vulnerable to this form of masquerading.
3. Partial Encryption or Low-Entropy Padding: To evade content-based detection, which often relies on observing high entropy values in encrypted files, attackers may encrypt only portions of files or pad them with low-entropy data. This technique aims to make the modified files appear less "malicious" from a statistical perspective.

The core problem, as articulated by the speaker, is that current detection systems are primarily designed for "bursty" or "high-entropy" indicators, making them susceptible to these evasive maneuvers. To address this, the research behind ERW-Radar began with a detailed observation of evasive ransomware's I/O behaviors. By collecting and running samples, the researchers identified a striking characteristic: repetitive behavior segments. Across various attack types, including splitting attacks (where encryption is spread across multiple processes or phases) and intermittent attacks (where encryption is paused and resumed), processes exhibited similar series of operations over extended periods. This led to the conclusion that evasive ransomware, despite its varied tactics, repetitively executes similar operations based on underlying templates, resulting in a "unique repetitive characteristic" in its I/O behavior over the long term.

Beyond behavioral patterns, the research also explored content-based indicators. Comparing encrypted files with benignly modified ones, the chi-square test showed promise in distinguishing between them. Further analysis of the probability distribution of bytes revealed distinct patterns: uniform distribution in truly encrypted files, slight fluctuations in files padded with low-entropy data, and numerous peaks in benignly modified files. Combining these two fine-grained content analysis techniques proved crucial for more accurately classifying files, even when entropy differences were not significant enough for traditional methods. These foundational observations form the bedrock upon which ERW-Radar's adaptive detection capabilities are built.

Key Findings

▶ Watch: Introducing ERW-Radar and its three main components (4:00)

The research underpinning ERW-Radar yielded several critical findings that collectively address the limitations of existing ransomware detection systems:

1. Repetitive I/O Characteristics in Evasive Ransomware: Despite adopting diverse evasive strategies such as splitting attacks or intermittent encryption, evasive ransomware consistently exhibits a "unique repetitive characteristic" in its I/O behavior over long periods. This stems from the repetitive execution of similar operations based on underlying templates, offering a consistent signature for detection.
2. Effectiveness of Fine-Grained Content Analysis: Traditional reliance on high entropy values for identifying encrypted files is insufficient against evasive techniques like partial encryption or low-entropy padding. ERW-Radar demonstrates that combining the chi-square test with the analysis of probability distribution of bytes provides significantly more accurate and "fine-grained" indicators. These methods can reliably distinguish between genuinely encrypted, padded, and benignly modified files, even when entropy differences are subtle.
3. Overhead Reduction through Customized I/O Extraction: The sheer volume and diversity of I/O operations (over 30 types of IOCTLs, up to 3,000 per second) and the overhead of kernel-to-user space context switching pose significant challenges for real-time analysis. A key finding is that a lightweight and customized information extraction solution can drastically reduce this burden by selectively parsing only file-related operations, filtering low-frequency events, and caching data for periodic transmission.
4. Correlation for Adaptive Behavioral Detection: Detecting repetitive patterns is complicated by system resource competition and inconsistent intervals between malicious operations. The research found that a correlation mechanism, which analyzes the relationship between recent and historical behavior segments, is superior to rigid rule-based or periodicity-focused solutions. This mechanism effectively identifies ongoing malicious operations by leveraging contextual references.
5. Dynamic Detection Window for Optimal Balance: A critical challenge is balancing detection efficiency (requiring a short analysis window) with the need for sufficient contextual information (requiring a longer window). The finding highlights the necessity of an adaptive or dynamic detection window that adjusts its size based on the consistency of results from both behavioral detection and content analysis, optimizing for both speed and accuracy.
6. User-Space Content Analysis at Idle Cycles: Performing intensive content analysis in the kernel space is resource-prohibitive. ERW-Radar's findings indicate that offloading fine-grained content analysis to the user space, and strategically triggering it during idle I/O cycles, provides an effective way to conduct thorough evaluations (chi-square, probability distribution) without impacting system performance. The concept of an "IO train" is introduced to sensitively identify these idle periods.

Technical Deep Dive

▶ Watch: Correlation mechanism for detecting behavioral repetitiveness (5:10)

ERW-Radar is architected with three primary components designed to synergistically address the challenges posed by evasive ransomware: a contextual behavior detector, a fine-grained content analyzer, and adaptive strategies that govern their interaction.

Lightweight and Customized Information Extraction

The first technical hurdle is the immense overhead associated with collecting and processing I/O behavior data. Modern operating systems generate a vast stream of IOCTLs (I/O control requests), with over 30 types and frequencies reaching up to 3,000 per second. Parsing all of these, especially when combined with frequent context switching between kernel and user space for information transmission, creates an unacceptable performance burden.

ERW-Radar's solution is a lightweight and customized information extraction mechanism. Instead of indiscriminately processing all IOCTLs, the system employs a filtering approach:

1. Selective Parsing: Most IOCTLs are allowed to pass through directly. Only file-related operations are targeted for detailed parsing, significantly reducing the data volume.
2. Frequency-Based Filtering: Operations with inherently low frequencies are further filtered out, focusing on those most relevant to potential malicious activity.
3. Batched Transmission: Extracted behavioral information is cached in a queue within the kernel space and then transmitted to user space periodically. This batching strategy minimizes the frequency of context switches, thereby alleviating system burden.

Accurate Detection of Behavioral Repetitiveness

Detecting the "unique repetitive characteristic" of evasive ransomware is complex because these repetitive behavior segments are not always perfectly consistent. System resource competition from other programs, especially those with heavy workloads, can introduce variability. Furthermore, the intervals between these segments are often inconsistent, rendering traditional rule-based or strict periodicity-focused detection methods ineffective.

ERW-Radar overcomes this with a novel correlation mechanism. The core insight is that recent behavior segments provide precise, real-time information about ongoing operations, while historical segments offer crucial contextual references. The correlation between these two aspects can accurately reflect behavioral repetitiveness. The mechanism operates by:

1. Subsequence Generation: The latest behavior sequences are broken down into progressive subsequences.
2. Sequence-Wise Analysis: Each of these latest subsequences is then subjected to a sequence-wise analysis against a repository of historical sequences. This comparison quantifies the similarity and correlation, allowing the system to identify subtle, recurring patterns even if they are not perfectly identical or strictly periodic.

Adaptive/Dynamic Detection Window

A critical design consideration for any real-time detection system is the size of its detection window. A short window offers high efficiency and low latency, but may lack sufficient contextual information to detect highly evasive, slow-paced threats. Conversely, a long detection window provides ample contextual data but incurs higher computational latency, potentially leading to greater file loss before detection.

ERW-Radar addresses this dilemma with a variable-length detection window that adapts dynamically based on the consistency of its two primary detection components:

1. Case 1: Consistent Positive Results: If both the behavioral detection (correlation mechanism) and the fine-grained content analysis consistently yield positive results, it indicates that the current window size is sufficient and effective in distinguishing malicious activity. In this scenario, the system considers shortening the window to a more "visible" size to detect ransomware faster and minimize potential damage.
2. Case 2: Inconsistent Results: If the results from the two components remain inconsistent (e.g., behavioral detection is positive, but content analysis is inconclusive, or vice versa), it suggests that the current window size might be too small. This could either make it difficult to identify malicious behaviors clearly or lead to an unacceptably high rate of false positives. In this situation, the system needs to expand the window to capture more contextual information, allowing for a more definitive conclusion.

Fine-grained Content Analysis at Idle Cycles

Existing content-based analysis often focuses on the write buffers of IOCTLs and is typically performed in the kernel space. This approach is severely limited by the kernel's constrained computing and storage resources, making it impractical to conduct computationally intensive operations like chi-square tests and probability distribution evaluations, or to store excessive content for accurate analysis.

ERW-Radar's innovative approach shifts content analysis to the user space, where computational and storage resources are more abundant. The insight is to analyze a series of file segments rather than individual write buffers.

1. Segment-Based Analysis: The system targets a series of segments of varying sizes from modified files.
2. Advanced Statistical Evaluation: These segments are then evaluated using two powerful, fine-grained indicators: the chi-square test and the probability distribution of bytes. These methods provide a more nuanced understanding of file modification than simple entropy values, effectively discerning encrypted content from benign changes or low-entropy padding.
3. Idle Cycle Triggering: To prevent this intensive analysis from impeding system performance, it is strategically triggered during idle I/O cycles. The system assesses the "I/O business" using an IO train mechanism. This mechanism is highly sensitive to the latest values of time intervals between I/O requests, recognizing that sustained high intervals imply an arrival of analysis time. This ensures that the deep content analysis is performed only when system resources are least contended, minimizing its impact.

The robust integration of these components allows ERW-Radar to offer an adaptive and highly accurate defense against even the most sophisticated evasive ransomware techniques.

Demo / Proof of Concept

▶ Watch: Adaptive detection window for balancing efficiency (6:07)

While the talk does not describe a live demonstration of ERW-Radar in action, it does present a comprehensive evaluation of its performance against a curated dataset. This evaluation serves as the proof of concept, showcasing the system's efficacy in detecting evasive ransomware in a controlled environment. The dataset included both ransomware I/O behaviors and benign I/O behaviors, allowing for a robust assessment of ERW-Radar's ability to distinguish malicious activity from legitimate system operations.

The speaker presented quantitative results highlighting ERW-Radar's improvements over existing solutions. Specifically, when compared to ShieldFS, a representative solution designed to counter evasive ransomware, ERW-Radar demonstrated a significant increase in recall—ranging from 6% to 27% in ransomware detection. Furthermore, when compared to a baseline using the original transformer architecture for its behavioral detection component, ERW-Radar achieved an improvement in overall accuracy of more than 5%. These metrics underscore the practical effectiveness of ERW-Radar's adaptive strategies and fine-grained analysis in enhancing detection capabilities while managing the cost of detection, including detection time and file data loss.

Defensive Implications

▶ Watch: Summary of ERW-Radar's challenges and solutions (9:00)

ERW-Radar's design and findings offer several critical implications for cybersecurity defenders looking to bolster their defenses against evasive ransomware:

1. Shift Beyond Static Signatures and Simple Thresholds: Defenders must recognize that traditional detection methods, relying on fixed signatures, high I/O volume, or simple entropy checks, are no longer sufficient. Evasive ransomware actively manipulates these indicators. Security solutions need to incorporate more dynamic and contextual analysis.
2. Embrace Long-Term Behavioral Analysis: The discovery of "unique repetitive characteristics" in evasive ransomware's I/O behavior highlights the importance of monitoring processes over extended periods. Defenders should seek or implement systems capable of correlating recent I/O activities with historical patterns, even if individual operations appear benign or are deliberately slowed down. This requires robust logging and analytical capabilities.
3. Adopt Fine-Grained Content Analysis: Relying solely on file entropy is a significant vulnerability. Security products should integrate advanced statistical methods like chi-square tests and probability distribution of bytes for content analysis. These techniques provide a deeper understanding of file modifications, enabling the distinction between genuine encryption, low-entropy padding, and legitimate changes. This is particularly crucial for protecting against partial encryption attacks.
4. Implement Adaptive Detection Logic: The concept of a dynamic detection window is vital. Defense systems should be capable of adjusting their analysis scope (time window) based on real-time feedback from multiple detection engines. This allows for rapid response when threats are clearly identified and extended observation when behaviors are ambiguous, optimizing both performance and detection accuracy.
5. Strategically Offload Resource-Intensive Tasks: Performing heavy computational analysis (like detailed content inspection) in the kernel can severely degrade system performance. Defenders should consider solutions that intelligently offload such tasks to user space and schedule them during idle I/O cycles. This ensures that critical analysis can occur without hindering system responsiveness, making the defense system practical for deployment in production environments.
6. Continuous Online Monitoring for Behavior: While content analysis can be deferred to idle cycles, the behavioral detection component should operate in an online mode, constantly monitoring process activities. This continuous vigilance is crucial for catching ransomware that attempts to slow down attacks or mimic benign programs, providing an immediate first line of defense.
7. Awareness of Threat Model Limitations: Defenders should be mindful of the specific threat model addressed by ERW-Radar. It focuses on ransomware that encrypts the majority of file content and does not account for ransomware that operates at extremely slow speeds (e.g., one byte per hour) or privileged attackers who might maliciously shut down detection systems. A layered defense strategy is always recommended to cover a broader spectrum of threats.

By integrating these implications into their security strategies, organizations can significantly enhance their resilience against the evolving landscape of evasive ransomware, moving towards more intelligent and adaptive defense mechanisms.

Key Takeaways

• Evasive ransomware bypasses traditional defenses by altering I/O patterns (reduced frequency, intermittent attacks), imitating benign programs, or using partial encryption and low-entropy padding, rendering static signatures and simple entropy checks ineffective.
• ERW-Radar employs a novel correlation mechanism to accurately detect repetitive I/O behaviors, even when they are inconsistent or deliberately slowed down, by analyzing the relationship between recent and historical process activities.
• A dynamic detection window is crucial for balancing detection efficiency and the need for sufficient contextual information, adapting its size based on the consistency of behavioral and content analysis results.
• Fine-grained content analysis, utilizing chi-square tests and the probability distribution of bytes, provides superior accuracy over simple entropy checks, enabling precise identification of encrypted, padded, or benignly modified files.
• Resource-intensive content analysis is performed in user space during idle I/O cycles to minimize overhead, while a lightweight, customized I/O extraction and online behavioral detection component ensure continuous monitoring without significant performance impact.
• Evaluation demonstrates ERW-Radar's effectiveness, significantly increasing recall by 6-27% compared to solutions like ShieldFS and improving accuracy by over 5% against a baseline transformer architecture, validating its adaptive and multi-faceted approach.

About the Speaker(s)

The talk was presented by Lingbo Zhao. Based on the provided metadata and transcript, no specific title or company affiliation was mentioned during the presentation or in the talk's details. Lingbo Zhao discussed the research behind ERW-Radar, an adaptive detection system against evasive ransomware.