DeepShuffle: A Lightweight Defense Framework against Adversarial Fault Injection Attacks on Deep Neural Networks in Multi-Tenant Cloud-FPGA

DeepShuffle: A Lightweight Defense Framework against Adversarial Fault Injection Attacks on Deep Neural Networks in Multi-Tenant Cloud-FPGA

Speakers: Yukui Luo, Assistant Professor, Umass (formerly Northeastern University); Adnan Siraj Rakin; Deliang Fan; Xiaolin Xu

Conference: IEEE S&P

YouTube: https://www.youtube.com/watch?v=aeijjStjc7w

Overview

This talk introduces DeepShuffle, a novel and lightweight defense framework designed to protect Deep Neural Networks (DNNs) from adversarial fault injection attacks within multi-tenant cloud-FPGA environments. Presented by Yukui Luo and his co-authors, the research addresses a critical security vulnerability emerging with the increasing adoption of Field-Programmable Gate Array (FPGA) virtualization in cloud computing. While CPU and GPU virtualization are well-established, FPGA virtualization is a nascent field, presenting unique attack surfaces due to shared hardware resources like power distribution networks.

The core motivation for DeepShuffle stems from prior work, DeepDup, which demonstrated the feasibility of degrading DNN performance by injecting precisely timed faults through supply voltage fluctuations. This paper expands on that threat model, investigating its impact on highly optimized, open-source DNN accelerators and proposing an innovative, low-overhead mitigation strategy. DeepShuffle aims to randomize the attack surface, making it significantly harder for adversaries to consistently inject faults and compromise DNN inference accuracy.

This work is particularly significant for cloud providers and users deploying AI/ML workloads on FPGAs, as it highlights a concrete threat and offers a practical, deployable defense. The framework's lightweight nature, coupled with its ability to operate without requiring DNN model retraining, makes it an attractive solution for enhancing the security and reliability of AI services in multi-tenant cloud environments.

Background

▶ Watch: Introduction to DeepShuffle and motivation for multi-tenant FPGA security (0:00)

Cloud computing has witnessed rapid expansion over the last decade, driven by the desire for increased resource utilization effectiveness. A significant component of this growth has been hardware virtualization, which allows multiple users or applications to share physical hardware resources efficiently. While virtualization technologies for CPUs and GPUs have matured considerably, FPGA virtualization in the cloud remains an emerging and active area of research. Current efforts in this domain primarily focus on enabling resource sharing and achieving multi-tenancy on a single FPGA chip, with several prototypes already implemented in both public and private clouds.

However, the hardware-programmable nature of FPGAs, especially in a multi-tenant context, introduces unique security challenges and attack surfaces. The sharing of critical hardware resources, such as the power distribution network, can facilitate indirect interactions between the circuit applications of different users. This shared environment opens the door for side-channel attacks and fault injection scenarios.

Previous work by the authors, termed DeepDup, demonstrated a potent adversarial fault injection attack. DeepDup leveraged supply voltage fluctuation and a side-channel guided attack controller to inject well-timed faults at runtime, specifically targeting and degrading the performance of DNN modules executing within a cloud FPGA context. This attack proved effective against manually designed DNN accelerators.

This current research aimed to address two pivotal questions:

1. Can DeepDup effectively target and degrade the performance of highly optimized, open-source DNN accelerators, given that previous demonstrations focused on manually crafted ones?
2. Is there a lightweight method to mitigate DeepDup-like adversarial fault injection attacks, particularly one that avoids the significant overhead of model retraining or extensive hardware modifications?

To answer the first question, the researchers selected TVM VTA (Versatile Tensor Accelerator) as their attack target. TVM VTA is an open-source DNN accelerator designed to run on FPGAs, providing a compiler to optimize the execution process of various DNN architectures for enhanced performance. Following a standard multi-tenant FPGA threat model, the team successfully reproduced both targeted and untargeted DeepDup attacks using ResNet-18 on the ImageNet dataset, executed on the TVM VTA accelerator.

Their findings revealed that TVM VTA exhibited greater robustness compared to the previously targeted manually-made accelerators. The primary reason for this increased resilience was identified: the TVM VTA offloads the execution of the first convolutional layer and the last fully connected layer to the ARM core, which operates outside the FPGA fabric. This architectural design choice inherently protects these critical layers from FPGA-specific fault injection attacks. Despite this, DeepDup could still achieve substantial accuracy degradation, establishing a new baseline for the attack's effectiveness against optimized open-source platforms.

Having confirmed the continued threat posed by DeepDup, the stage was set to address the second, more critical question: developing a lightweight mitigation strategy.

Key Findings

▶ Watch: DeepShuffle's core idea: shuffling weights as moving target defense (4:00)

The research yielded several significant findings, establishing both the continued threat of adversarial fault injection on optimized DNN accelerators and the efficacy of DeepShuffle as a defense.

First, the study confirmed that even highly optimized, open-source DNN accelerators like TVM VTA are vulnerable to DeepDup-style adversarial fault injection attacks, albeit with greater robustness than manually designed ones. This validated the necessity for robust defense mechanisms in cloud-FPGA environments. The observed resilience of TVM VTA was attributed to its architecture, where the first convolutional layer and the last fully connected layer are executed on the ARM core outside the FPGA, thus being inherently protected from FPGA-specific fault injections.

The central and most impactful finding is the development and validation of DeepShuffle as a lightweight and highly effective mitigation strategy. The core concept behind DeepShuffle is to implement a moving target defense by shuffling the weight transmission order of DNN parameters during inference. This randomization makes it exceedingly difficult for an adversary, such as DeepDup, to consistently identify and exploit a "favorable weight transmission point" – a specific data package whose corruption would most significantly impact overall inference accuracy.

DeepShuffle operates based on three fundamental principles for effective weight transmission shuffling:

1. Principle 1: For a convolutional 2D layer, shuffling the weight parameters along the output channel will accordingly shuffle the output feature maps of that layer.
2. Principle 2: To maintain computational correctness throughout the DNN inference process, if active channel shuffling is applied to a convolutional 2D layer using a specific rule, the same shuffling rule must also be applied to its connected layers (e.g., subsequent convolutional layers, Batch Normalization (BN) layers). Crucially, this is achieved without requiring any additional operators or computational overhead.
3. Principle 3: Principles one and two are broadly applicable to all layers with trainable parameters within a DNN architecture, including Residual Blocks and Dense Blocks.

Through extensive evaluation, DeepShuffle demonstrated a significant increase in the effort required for DeepDup to succeed. This effort is quantified by the DeepDup Pering Index (DPI), an evaluation metric that includes parameters like the number of evolutions, mutation numbers, attack rounds, sample numbers, and input batch size. DeepShuffle substantially increased the DPI compared to the baseline, indicating that DeepDup needed significantly more inference rounds to complete its search for effective attack points.

Furthermore, DeepShuffle proved highly effective in maintaining DNN accuracy even under prolonged attack. For untargeted attacks, it preserved substantial inference accuracy even after 400 rounds of attack. In targeted attack scenarios, where DeepDup aims to misclassify specific samples into a chosen target class, DeepShuffle ensured that the targeted class could still be detected even after 1,000 rounds of continuous attack, significantly mitigating the threat of successful misclassification.

Finally, a detailed layer-wise robustness analysis revealed that while DeepDup could reduce post-attack accuracy to as low as 10% for various layers in ResNet-20, DeepShuffle dramatically improved the robustness of particularly vulnerable layers, such as layer 4, layer 7, and the last layer. This targeted improvement underscores DeepShuffle's ability to fortify specific weak points within DNN architectures. Crucially, DeepShuffle achieves all these benefits without requiring any retraining of the DNN model, making it a highly practical and deployment-friendly solution applicable across diverse domains like computer vision, natural language processing, and robotics.

Technical Deep Dive

▶ Watch: Visualizing DeepShuffle's defense against DeepDup attacks (6:00)

The technical ingenuity of DeepShuffle lies in its elegant solution to a complex problem: how to disrupt an adversary's ability to consistently exploit specific data patterns without incurring significant performance penalties or requiring model retraining. The DeepDup attack operates by identifying and corrupting specific weight transmission packages that, when perturbed at precise moments, lead to the most substantial degradation in overall inference accuracy. In a normal DNN inference, the sequence of these weight transmissions is fixed, allowing an attacker to learn and repeatedly exploit these critical points.

DeepShuffle counters this by introducing a moving target defense through the dynamic randomization of the weight transmission order. The core idea is to ensure that the "favorable attack point" identified by DeepDup in one inference run becomes ineffective or significantly less potent in subsequent runs due to the altered data flow.

The implementation of this randomization is guided by three fundamental principles rooted in the inherent properties of DNNs:

1. Principle 1: Output Channel Shuffling: For a convolutional 2D layer, the weights are typically organized by output channels. If the weight parameters are shuffled along their output channel dimension, the corresponding output feature maps generated by that layer will also be shuffled accordingly. This effectively changes the internal representation and flow of information.

1. Principle 2: Connected Layer Consistency: To maintain computational correctness across the entire DNN, if a specific shuffling rule is applied to a convolutional layer, the same rule must be consistently applied to all subsequent, connected layers that process its output. This includes layers like Batch Normalization (BN) layers and subsequent convolutional layers. The beauty of this principle is that it allows the shuffling to propagate through the network without needing additional operators or modifying the underlying computational graph, thus avoiding performance overhead. For example, if the output channels of a convolutional layer are permuted, the corresponding input channels of the next layer must be permuted by the same rule, and the scaling/shifting parameters of an intermediate BN layer must also be permuted.

1. Principle 3: Universal Applicability: These shuffling principles are not limited to simple convolutional layers. They can be applied to all layers with trainable parameters within a DNN architecture. The talk provides examples of how to apply these shuffling rules to more complex structures such as Batch Normalization layers, Residual Blocks (including those with downsampling layers), and Dense Blocks. This broad applicability ensures that DeepShuffle can be integrated into a wide range of modern DNN architectures.

A visual demonstration helps to clarify the process: in a typical DNN accelerator, weights are buffered from onboard memory to an on-chip buffer for computation. DeepDup targets moments when a specific weight package interacts with its corresponding feature map, especially those containing significant information about the input image, leading to effective attacks. DeepShuffle disrupts this by continuously changing the attack point. In every inference, the specific memory locations or data streams that were critical for DeepDup in the previous round are no longer the same, making it challenging for the attacker to consistently target the most impactful data.

DeepDup's attack strategy relies on a progressive differential evolutional search algorithm. This algorithm initializes a search space and iteratively evaluates a fitness function (e.g., inference loss) to identify optimal fault injection points. The algorithm typically calculates a table mapping weight transmission IDs to their corresponding loss values when attacked. DeepShuffle intervenes at the third step of this algorithm, which involves target selection. By continuously moving the target – randomizing the weight transmission order – DeepShuffle ensures that the attacker's search algorithm cannot yield consistent results across iterations. Each search iteration effectively starts from a new, randomized state, significantly increasing the time and effort required for DeepDup to find an effective attack point.

To quantify this increased effort, the researchers introduced the DeepDup Pering Index (DPI). This metric encapsulates various factors of DeepDup's search process, including the number of evolutions, mutation numbers, number of attack rounds, sample numbers, and input batch size. A higher DPI indicates that DeepDup needs more inference rounds to complete its search and identify an effective attack. The experimental results, conducted on two different datasets and five different DNN accelerators, clearly showed that DeepShuffle dramatically increased the DPI compared to the baseline, thereby imposing a substantial computational burden on the attacker.

Further technical analysis involved feature similarity analysis. By comparing the output feature maps of the second convolutional 2D layer of a ResNet-18 model under normal inference, DeepDup attack, and DeepShuffle defense, the team observed striking differences. Under DeepDup, the output feature maps became significantly different from the original, indicating severe corruption. However, when DeepShuffle was applied, it effectively mitigated such changes, maintaining a high degree of similarity to the original output and thus preserving the integrity of the intermediate representations.

Finally, a layer-wise robustness test was performed on ResNet-20 to identify which layers were most vulnerable and how DeepShuffle improved their resilience. Without DeepShuffle, an attacker could cause a drop in post-attack accuracy to as low as 10% regardless of the layer targeted, with specific layers (e.g., layer 4, layer 7, and the last layer) being highly vulnerable, requiring only around 30 attack rounds to achieve a random guess goal. DeepShuffle significantly improved the robustness of these vulnerable layers, demonstrating its ability to fortify critical parts of the DNN architecture against targeted fault injection.

Demo / Proof of Concept

▶ Watch: Examples of feature map degradation caused by DeepDup (7:45)

While the talk did not feature a live, interactive demonstration, the research thoroughly validated the effectiveness of DeepShuffle through extensive empirical evaluation and experimental results. The "proof of concept" was established by systematically reproducing DeepDup attacks on a challenging, optimized target and then demonstrating DeepShuffle's mitigation capabilities across various scenarios.

The primary target for the attacks and subsequent defense evaluation was TVM VTA, an open-source DNN accelerator running on an FPGA. This choice was deliberate, as previous DeepDup demonstrations focused on less optimized, manually designed accelerators, and the researchers sought to confirm the threat against more robust, real-world platforms.

The experimental setup involved:

• Untargeted Attacks: Using ResNet-18 on the ImageNet dataset. The goal was to degrade overall inference accuracy without aiming for a specific misclassification.
• Targeted Attacks: Employing a custom dataset based on CIFAR-10, with Class 8 selected as the specific target for misclassification. Targeted attacks are often considered more potent as they aim to manipulate the model's output in a predictable, malicious way.

The evaluation leveraged the DeepDup Pering Index (DPI), a comprehensive metric designed to quantify the effort an attacker needs to find an effective fault injection point. The experiments clearly showed that DeepShuffle significantly increased the DPI for DeepDup across different datasets and DNN accelerators, proving that the defense forces the attacker to expend considerably more resources and time to achieve their objectives.

Quantitative results highlighted DeepShuffle's strong performance:

• Untargeted Attack Mitigation: After 400 rounds of continuous DeepDup attacks, a DNN accelerator protected by DeepShuffle was still able to maintain substantial inference accuracy. This contrasts sharply with unprotected systems where accuracy would rapidly degrade.
• Targeted Attack Mitigation: Even after an astonishing 1,000 rounds of targeted DeepDup attacks, DeepShuffle successfully ensured that the targeted class could still be correctly detected. This demonstrates a robust defense against sophisticated adversaries aiming for specific misclassifications.
• Layer-Specific Robustness: A detailed analysis of ResNet-20 revealed that certain layers (e.g., layer 4, layer 7, and the last layer) were particularly vulnerable to DeepDup, with attacks capable of reducing accuracy to a mere 10% after only 30 rounds. DeepShuffle dramatically improved the robustness of these specific layers, preventing such severe accuracy drops and showcasing its ability to fortify critical components of the DNN.

These results collectively served as a compelling proof of concept for DeepShuffle. They empirically validated its ability to randomize attack opportunities, significantly increase attacker effort, and preserve the integrity and accuracy of DNN inference in multi-tenant cloud-FPGA environments, all without the need for computationally expensive model retraining.

Defensive Implications

▶ Watch: DeepShuffle's evaluation results: increased DPI and maintained accuracy (10:10)

The findings presented in the DeepShuffle talk carry significant defensive implications for anyone deploying or managing Deep Neural Networks (DNNs) on multi-tenant cloud-FPGA platforms. As FPGA virtualization continues to mature and integrate into mainstream cloud offerings, understanding and mitigating unique attack surfaces like adversarial fault injection becomes paramount.

Firstly, cloud providers and users must recognize the inherent security risks associated with shared hardware resources in FPGA environments. The DeepDup attack serves as a concrete example of how subtle interactions, such as those through the power distribution network, can be exploited to compromise critical AI workloads. This underscores the need for security-by-design principles in future FPGA virtualization architectures.

The primary defensive implication is the immediate applicability of DeepShuffle as a lightweight and effective countermeasure. Organizations should consider integrating DeepShuffle's principles into their DNN deployment pipelines on FPGAs. Specifically, this involves implementing the dynamic randomization of weight transmission order based on the three principles outlined: shuffling along the output channel, ensuring computational correctness across connected layers, and applying this broadly to all layers with trainable parameters. Since DeepShuffle does not require model retraining, its adoption cost is minimal, making it a highly practical defense to deploy in existing systems.

Furthermore, the layer-wise robustness analysis provides actionable intelligence for defenders. Identifying specific vulnerable layers (e.g., layer 4, layer 7, and the last layer in ResNet-20) allows for targeted hardening strategies. While DeepShuffle significantly improves their robustness, these layers might still represent residual weak points. The authors suggested future work could involve redesigning these particularly vulnerable layers or offloading their computation to more secure environments, such as the ARM core (as seen in TVM VTA's inherent robustness for its first and last layers) or even dedicated CPU resources, if performance constraints allow. This layered defense approach, combining DeepShuffle's randomization with architectural hardening of critical components, could offer even greater resilience.

DeepShuffle's nature as a moving target defense is a powerful paradigm shift. Instead of relying on static defenses that attackers can eventually bypass, it continuously alters the attack surface, increasing the attacker's DeepDup Pering Index (DPI) and forcing them to expend significantly more resources and time for each successful fault injection. This makes attacks less economically viable and harder to execute consistently.

Finally, the broad applicability of DeepShuffle across various domains (computer vision, natural language processing, robotics) suggests that this defense mechanism can protect a wide array of AI services. Defenders should evaluate their specific DNN workloads running on FPGAs and assess how DeepShuffle could be integrated to bolster their security posture against emerging hardware-level attacks.

Key Takeaways

• DeepShuffle is a lightweight defense framework specifically designed to protect Deep Neural Networks (DNNs) from adversarial fault injection attacks in multi-tenant cloud-FPGA environments.
• The framework implements a moving target defense by dynamically shuffling the weight transmission order during DNN inference, making it significantly harder for attackers like DeepDup to consistently find and exploit critical fault injection points.
• DeepShuffle operates without requiring any retraining of the DNN model, making it a practical and low-overhead solution for immediate deployment.
• It significantly increases the attacker's effort, quantified by the DeepDup Pering Index (DPI), demonstrating its effectiveness in making fault injection attacks less efficient and more resource-intensive for adversaries.
• The defense demonstrably maintains high inference accuracy even after hundreds or thousands of attack rounds, mitigating both untargeted and targeted misclassification attempts.
• DeepShuffle improves the robustness of particularly vulnerable layers within DNN architectures (e.g., layer 4, 7, and the last layer in ResNet-20), providing targeted protection where it's most needed.

About the Speaker(s)

The primary presenter for this talk was Yukui Luo. At the time of the presentation, Yukui Luo had graduated from Northeastern University and was working as an Assistant Professor at Umass. The research was a collaborative effort with co-authors Adnan Siraj Rakin, Deliang Fan, and Xiaolin Xu.