A New PPML Paradigm for Quantized Models

A New PPML Paradigm for Quantized Models

Speakers: Tianpei Lu (presented by Bingshan from Jan University)

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=7gXbEtxp4Qk

Overview

This talk introduces a groundbreaking new paradigm for Privacy-Preserving Machine Learning (PPML) specifically tailored for quantized models. Presented by Bingshan from Jan University, the work, a collaboration with Tianpei Lu, Shiaan, and Quiran, addresses a critical gap in the secure computation landscape. The core challenge in PPML, particularly for model inference where a server holds a private model and a client holds private data, has long been the prohibitive computational cost associated with cryptographic techniques like multi-party computation (MPC). While existing PPML solutions strive to protect the confidentiality of both the model weights (w) and the client's input data (x) during the computation of M(w, x), they often struggle with the inherent complexities of floating-point or fixed-point arithmetic.

The presentation highlights that traditional PPML approaches, when dealing with numerical operations, rely heavily on fixed-point arithmetic. This method, while necessary for precision, introduces significant overhead, primarily due to the need for expensive truncation operations. Truncation, which involves scaling values up for multiplication and then scaling them back down, is computationally intensive and prone to errors if not handled meticulously, often requiring the allocation of "extra space" that further degrades efficiency. This problem is exacerbated when attempting to apply these techniques to quantized models, which are increasingly prevalent in real-world applications due to their efficiency in terms of memory, storage, and computational demands, especially on resource-constrained devices.

The central contribution of this research is the proposition of a novel PPML architecture that leverages Lookup Tables (LUTs) to execute all model operations. By representing both linear and non-linear functions within a model as lookup tables, the authors eliminate the need for costly fixed-point truncation. This approach is particularly well-suited for quantized models, where inputs and outputs are typically represented with a small number of bits (e.g., 8-bit integers), resulting in manageable table sizes (e.g., 256 entries). The result is a dramatically more efficient PPML solution that achieves significant speedups compared to state-of-the-art methods, making privacy-preserving inference for practical machine learning models a far more viable reality.

Background

▶ Watch: Introduction to privacy-preserving machine learning (PPML) (0:00)

The concept of Privacy-Preserving Machine Learning (PPML) is rooted in the necessity to perform computations on sensitive data without revealing the data itself or the underlying model parameters. In the typical scenario discussed, a server possesses a proprietary machine learning model (its weights w), and a client holds private input data (x), such as a prompt for a generative AI. The objective is to jointly compute the model's output y = M(w, x) while ensuring that the server never learns x and the client never learns w. This is fundamentally a problem of secure multi-party computation (MPC), although other techniques like Fully Homomorphic Encryption (FHE) or Trusted Execution Environments (TEEs) are also explored in the broader literature. This specific work focuses on a two-party computation (2PC) setting, which simplifies certain aspects but still faces considerable cryptographic challenges.

A primary hurdle in existing PPML frameworks, especially those based on MPC, lies in the handling of numerical precision. Most machine learning models operate with floating-point numbers. However, performing floating-point arithmetic securely within MPC protocols is exceptionally complex and computationally expensive. Consequently, PPML often resorts to fixed-point arithmetic, where numbers are represented as integers with a predefined number of bits allocated for the integer part and the fractional part. While more amenable to MPC, fixed-point arithmetic introduces its own set of problems. When two fixed-point numbers are multiplied, both their integer and fractional parts can expand, potentially exceeding the allocated bit-width. To maintain a consistent representation, the result must be scaled down, a process known as truncation.

Truncation, in essence, involves dividing by a scaling factor. In an MPC setting, this division must be performed securely, which is far from trivial. The speaker explains that in practice, to prevent loss of significant bits (which would lead to computational failure and disastrous accuracy loss), an "extra space" of leading zeros must be reserved. This padding, while crucial for correctness, directly impacts efficiency by requiring more bits to be processed securely. The cost of truncation is substantial: it can be implemented deterministically using complex circuits like Garbled Circuits (GC), which are notoriously expensive, or probabilistically, which introduces a controlled error rate (2^(1-L') for L' extra bits) but is still computationally demanding. These truncation operations are typically performed over rings (e.g., modulo 2^32 or 2^64) to align with computer architecture and leverage efficient integer operations.

The specific challenge that this research tackles emerges when considering quantized models. Quantized models are a prevalent optimization technique in modern machine learning, designed to reduce memory footprint, storage requirements, and computational load, particularly for deployment on edge devices or in scenarios with limited I/O bandwidth. Instead of using high-precision floating-point numbers (e.g., FP32 or FP64), quantized models represent weights and activations using lower-bit integers (e.g., 8-bit integers, or even 4-bit). In standard (non-PPML) operation, these int8 values are often "dequantized" to higher precision (e.g., FP32) inside a GPU for computation, and then "re-quantized" back to int8 before output.

The problem for PPML is that this dequantization and requantization process involves scalar rescaling factors (e.g., sx * sy / sz), which are fractional numbers. Applying these factors within an MPC environment for quantized models effectively reintroduces the need for complex fixed-point arithmetic and, critically, the expensive truncation operations that existing PPML struggles with. Thus, despite the inherent efficiency of quantized models in plaintext, their secure computation within PPML environments had remained an open and challenging problem due to the fundamental mismatch with existing MPC primitives.

Key Findings

▶ Watch: Explaining the high cost of truncation in MPC (4:00)

The central innovation presented in this talk is a novel paradigm for Privacy-Preserving Machine Learning (PPML) that fundamentally rethinks how operations within quantized models are executed securely. The core insight is to replace complex fixed-point arithmetic and its associated expensive truncation with Lookup Tables (LUTs) for all model computations. This includes not only non-linear activation functions but also linear operations like matrix multiplications, which are typically the most computationally intensive parts of deep learning models.

The rationale behind using LUTs is elegantly simple and directly addresses the limitations of quantized models. Since quantized models operate with low-bit integer representations for both their inputs and outputs (e.g., 8-bit integers), the domain and range of any given function within the model are inherently constrained. For an 8-bit input, there are only 2^8 = 256 possible input values. This means that any function, regardless of its internal complexity, can be pre-computed and stored in a table with a maximum of 256 entries. This small table size is the key enabler for the proposed paradigm.

The use of LUTs brings several significant advantages:

1. Elimination of Truncation: By representing functions as pre-computed tables, the need for dynamic fixed-point arithmetic and its costly truncation operations is completely removed. The output is simply "looked up," bypassing the entire fixed-point precision management problem that plagues traditional PPML.
2. Hiding Internal Complexity: The internal mathematical complexity of the function is abstracted away within the table. The secure computation protocol only needs to perform an oblivious table lookup, not a secure evaluation of the function's arithmetic components.
3. Highly Efficient Online Phase: The most remarkable benefit is the drastic reduction in online computation and communication. Once the tables are set up in an offline phase, the online phase merely involves securely opening an index into the table. The speaker states that the online communication is effectively O(1) in a cryptographic sense, or O(log N) in an information-theoretic sense, meaning it's almost equivalent to sending plaintext data. This drastically reduces latency and bandwidth requirements during real-time inference.

To facilitate this, the researchers designed a bespoke Lookup Table evaluation protocol. This protocol enables a client to query a function f (known to the server) with their private input x without revealing x to the server or f to the client. The protocol involves an offline phase where the server and client collaboratively and obliviously permute and share the lookup table, and an online phase where the client simply provides an index to retrieve the result. This innovative approach effectively side-steps the long-standing efficiency bottlenecks in PPML for quantized models, offering a practical path toward widespread adoption.

Technical Deep Dive

▶ Watch: Overview of quantized models and their practical use (6:00)

The core of this new PPML paradigm lies in its innovative Lookup Table (LUT) evaluation protocol. This protocol is designed for a two-party setting where a server holds a private function f (representing a quantized model operation) and a client holds a private input x. The goal is for the client to learn f(x) without revealing x to the server or f to the client.

The protocol operates in two distinct phases:

1. Offline Phase:

This phase is computationally intensive but can be performed once in advance, prior to any live inference requests. The primary goal is to transform the original lookup table F (known to the server) into a securely shared, obliviously shifted version between the server and the client.

• Oblivious Cyclic Shift: The most critical component of the offline phase is an oblivious cyclic shift of the table. The speaker notes that existing work for oblivious shuffling of shared data exists, but their protocol is a novel modification that achieves an N times faster oblivious cyclic shift. This process ensures that the entries of the lookup table are permuted in a specific way, and this permutation (the shift amount R) is also shared secretly between the parties.
• Underlying Cryptographic Primitives: The protocol for the oblivious cyclic shift relies on advanced cryptographic primitives. Specifically, it requires N-1 out of N random Oblivious Transfers (OTs), which can be instantiated using techniques like the Goldreich-Micali-Wigderson (GGM) construction. While the speaker skips the intricate details due to time constraints, they illustrate a conceptual flow:
• The parties perform column-wise OTs.
• The client "misses" certain columns (represented as "red columns" in the presentation), while the server holds all data ("white columns").
• The data is cyclically shifted row by row (e.g., first row shifted by 0, second by 1, third by 2, etc.) to spread the "missing" elements across various positions.
• Intermediate sums (U for rows, V for columns) are computed securely.
• Output of Offline Phase: At the culmination of the offline phase, the server and client each hold a share of the cyclically shifted table and a share of the offset R. Critically, neither party individually knows the original table, the exact shift applied, or the other party's share.

1. Online Phase:

This phase is extremely lightweight and is executed when the client wishes to obtain an inference result.

• Index Opening: The client, possessing the private input x and their share of the offset R, computes x - R (modulo the table size). This value (x - R) is then revealed and used as an index.
• Shared Result Retrieval: Both the server and the client, using the now public index (x - R), retrieve the corresponding entry from their respective shares of the shifted table. The sum of these retrieved shares constitutes the final result f(x).
• Efficiency: The online communication is minimal, consisting only of revealing the index x - R. This makes the online phase highly efficient, approaching the speed of plaintext computation, as highlighted by the O(1) (cryptographic sense) or O(log N) (information theoretical sense) communication complexity.

Optimizations for Model Integration:

To make this LUT-based paradigm practical for complex machine learning models, the researchers introduced several crucial optimizations:

• Batching: Real-world ML inference often involves applying the same input x with multiple different parameters (e.g., W0, W1, W2). The protocol allows for batching these operations, where a single LUT can be used to handle multiple scalers with the same input x, achieving almost identical efficiency as a single operation. This is vital for layers like fully connected layers or convolutions with multiple filters.
• Operator Fusion: One of the most powerful optimizations is the ability to fuse multiple sequential layers or operations into a single, larger lookup table. For instance, a ReLU activation layer, followed by a Batch Normalization (BN) layer, and then a convolutional layer (or parts thereof) can all be combined into one composite lookup table. This convolution prime table effectively computes the output of the entire sequence of operations in a single lookup. This reduces the number of expensive online interactions and leverages the fact that the output of one quantized function serves as the input to the next, remaining within the low-bit integer domain. This fusion mechanism drastically improves overall efficiency by minimizing the overhead associated with chaining multiple secure computations.

These technical details underscore how the proposed paradigm not only bypasses the truncation problem but also incorporates practical optimizations that make it suitable for real-world quantized neural network architectures.

Demo / Proof of Concept

▶ Watch: Introducing lookup tables as the core solution (9:00)

While the talk did not feature a live, interactive demonstration of the system in action, the effectiveness and practicality of the proposed Lookup Table (LUT) paradigm for Privacy-Preserving Machine Learning (PPML) for quantized models were robustly validated through extensive performance benchmarks and accuracy evaluations. These results serve as the primary "proof of concept," demonstrating the significant efficiency gains and maintained model accuracy.

The performance evaluations focused on the speedup achieved over existing state-of-the-art PPML solutions:

• Dramatic Speedup: The new paradigm achieves remarkable speed improvements. For 8-bit quantization, the system is approximately 10 times faster than leading 3PC (three-party computation) and 2PC (two-party computation) solutions based on Function Secret Sharing (FSS), as well as those utilizing Garbled Circuits.
• LAN vs. WAN Performance: The distinction between LAN (Local Area Network) and WAN (Wide Area Network) settings is crucial, as communication latency is a major factor in MPC.
• In a LAN setting, the performance is already substantially better, with the specific figures detailed in the accompanying paper.
• In a WAN setting, where network latency is higher, the advantages of the O(1) online communication become even more pronounced. The system can be up to 1,000 times faster than traditional methods. This is attributed to the online phase merely requiring the opening of an index, making it akin to sending plaintext data, with minimal cryptographic overhead during real-time inference.
• Overall Efficiency: The speaker summarized that the approach achieves an overall speedup of 20 to 80 times compared to "all sort PPML" techniques, indicating a broad advantage across various comparison points.
• CPU vs. GPU: Notably, the proposed CPU-based solution was shown to be faster than even GPU-accelerated PPML frameworks like Sigma or Krypton. This highlights that the fundamental architectural shift to LUTs provides such a significant advantage that it can outperform hardware-accelerated traditional methods.

Regarding accuracy, the presentation clarified a crucial point: the research does not aim to improve the inherent accuracy of quantized models themselves, but rather to enable their privacy-preserving execution. The authors utilized already proposed quantized model parameters from existing literature and industry practices. These pre-quantized models are known to maintain "almost the same" accuracy as their non-quantized counterparts, a fact already leveraged by the industry for deployment. Therefore, by securely executing these optimized quantized models, the proposed PPML paradigm ensures that the accuracy benefits of quantization are preserved within the secure computation environment. This confirms that the efficiency gains do not come at the cost of model prediction quality.

These comprehensive performance figures and the clarity on accuracy demonstrate that the LUT-based PPML paradigm is not just theoretically sound but also practically viable, offering substantial improvements that could pave the way for wider adoption of secure machine learning inference.

Defensive Implications

▶ Watch: Key advantage: Eliminating expensive truncation with LUTs (10:00)

The introduction of a new PPML paradigm for quantized models carries significant defensive implications, offering practical advancements for securing machine learning deployments:

1. Enabling Secure Deployment of Quantized Models: Quantized models are ubiquitous in modern AI, especially in resource-constrained environments like mobile devices, edge computing, and large-scale cloud inference where efficiency is paramount. Prior to this work, deploying these models in a privacy-preserving manner was largely impractical due to the high computational cost of existing PPML techniques. This new LUT-based paradigm makes it feasible to run inference on these efficient models while protecting both the model's intellectual property and the client's sensitive input data.
2. Protecting Model Intellectual Property: For model owners (e.g., companies providing AI as a service), the model weights (w) represent significant intellectual property. This framework ensures that clients can submit data and receive predictions without ever seeing the proprietary model weights. This is a critical defense against model extraction attacks or unauthorized replication of valuable AI assets.
3. Safeguarding Client Data Privacy: Conversely, clients using these AI services often have highly sensitive input data (x), such as personal health information, financial data, or confidential enterprise documents. The PPML solution guarantees that the server performing the inference never directly accesses or learns the client's raw data. This is a fundamental defense against data breaches, unauthorized data usage, and compliance violations (e.g., GDPR, HIPAA).
4. Reducing Attack Surface and Complexity: By eliminating the need for complex fixed-point arithmetic and its associated expensive truncation operations, the underlying cryptographic protocols become simpler and more efficient. While the LUT evaluation protocol itself is sophisticated, the overall reduction in dynamic secure arithmetic operations can simplify the security analysis and potentially reduce the attack surface associated with complex arithmetic circuits.
5. Practical Adoption of PPML: The dramatic speedups (20-80x overall, up to 1000x in WAN settings) and the ability to operate efficiently on CPUs (outperforming GPU-accelerated traditional methods) significantly lower the barrier to entry for PPML. This makes privacy-preserving inference a more realistic option for a broader range of applications and organizations, moving it from a theoretical ideal to a deployable solution. Defenders can now advocate for and implement PPML in scenarios where it was previously deemed too slow or resource-intensive.
6. Future-Proofing AI Systems: As AI models become even more powerful and sensitive data is increasingly used, privacy-preserving techniques will become a standard requirement. This work pushes the frontier of practical PPML, offering a robust foundation for building secure and compliant AI systems that can scale with evolving threats and regulatory demands.

In essence, this research provides a powerful new tool in the defender's arsenal, enabling the secure and efficient deployment of the very AI models that are becoming indispensable across industries, thereby balancing innovation with robust data protection.

Key Takeaways

• Existing Privacy-Preserving Machine Learning (PPML) solutions struggle to efficiently support quantized models due to the high computational cost of fixed-point arithmetic truncation.
• This work introduces a novel PPML paradigm that represents all model operations (linear and non-linear) as Lookup Tables (LUTs), effectively eliminating the need for expensive truncation.
• A bespoke LUT evaluation protocol was developed, featuring an offline phase for oblivious table shifting and an extremely efficient online phase with O(1) communication for retrieving results.
• The paradigm incorporates critical optimizations such as batching and operator fusion (e.g., fusing ReLU, BN, and convolution into a single convolution prime LUT) to further enhance practical efficiency.
• Performance benchmarks show significant speedups: approximately 10 times faster in LAN settings and up to 1,000 times faster in WAN settings compared to state-of-the-art FSS and Garbled Circuit-based PPML.
• The CPU-based solution even outperforms GPU-accelerated PPML frameworks like Sigma or Krypton, demonstrating an overall 20-80 times speedup across various PPML comparisons.
• The approach maintains model accuracy by leveraging already optimized and industry-adopted quantized model parameters, ensuring that efficiency gains do not compromise prediction quality.

About the Speaker(s)

The talk "A New PPML Paradigm for Quantized Models" was presented by Bingshan from Jan University. He introduced the work as a joint effort with his PhD student Tianpei Lu, along with Shiaan and his "boss" Quiran. While the primary speaker was Bingshan, the research is a collaborative endeavor highlighting contributions from multiple individuals at Jan University, with Tianpei Lu likely being the lead author given the context of a PhD student's work. The presentation reflects expertise in applied cryptography, privacy-preserving machine learning, and the practical challenges of deploying secure AI models.