Manifoldchain: Maximizing Blockchain Throughput via Bandwidth-Clustered Sharding

Manifoldchain: Maximizing Blockchain Throughput via Bandwidth-Clustered Sharding

Speakers: Chunjiang Che

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=V4OUyCah8SM

Overview

In a presentation at the NDSS Symposium, researcher Chunjiang Che introduced Manifoldchain, a novel sharding protocol designed to significantly enhance blockchain throughput. The talk, titled "Manifoldchain: Maximizing Blockchain Throughput via Bandwidth-Clustered Sharding," addresses the persistent challenge of low transaction processing speeds in decentralized networks, a fundamental limitation hindering widespread blockchain adoption. Compared to centralized payment systems like Visa, which can handle over 24,000 transactions per second (TPS), current blockchain applications often struggle to exceed 20 TPS. This stark disparity underscores the urgent need for scalable solutions.

Manifoldchain directly confronts two primary bottlenecks responsible for this low throughput. First, blockchain systems typically involve highly overlapping tasks among miners—including consensus, ledger replication, computation, and storage—which inherently limit scalability with an increasing number of participants. Second, the "struggler problem" dictates that faster miners are often forced to wait for slower participants to synchronize, effectively bottlenecking the entire network's performance. While existing sharding protocols attempt to distribute workload, they often fall short by uniformly distributing miners, inadvertently scattering strugglers across all shards and thus limiting each shard's potential.

Manifoldchain differentiates itself by proposing a bandwidth-clustered sharding formation mechanism. This innovative approach groups miners with similar network bandwidth characteristics into dedicated shards, effectively separating high-performance miners from lower-performance ones. This segregation allows shards composed solely of fast miners to achieve substantially higher mining rates, thereby boosting overall network throughput. The protocol also introduces sophisticated mechanisms to maintain security in this non-uniform environment, where adversaries might otherwise exploit the clustered architecture.

Background

▶ Watch: Introduction and motivation for blockchain throughput (0:00)

The motivation behind Manifoldchain stems from the well-documented scalability crisis facing blockchain applications. Despite attracting immense attention for their decentralized and secure nature, these systems have consistently underperformed traditional centralized infrastructures in terms of transaction throughput. This performance gap is primarily attributed to two interconnected architectural bottlenecks inherent in many contemporary blockchain designs.

The first bottleneck arises from the highly redundant and overlapping tasks executed by network participants, particularly miners. In a typical blockchain, every miner must reach a consensus on the state of the ledger, replicate the entire transaction history, and perform identical computations and storage operations. This design, while fundamental to decentralization and security, means that the system's capacity does not scale linearly, or even significantly, with an increase in the number of miners. Adding more miners primarily enhances security and decentralization, but does little to alleviate the computational and communication burden, leading to a plateau in throughput regardless of network size.

The second, and often overlooked, bottleneck is the "struggler problem." In a network where all participants must eventually synchronize to a common state, the speed of the entire system is often dictated by its slowest components. Fast miners, capable of processing transactions and proposing blocks rapidly, are frequently compelled to wait for slower, less performant miners to catch up. This synchronization requirement effectively caps the overall throughput, as the system's pace is limited by its weakest links, preventing the full utilization of the network's collective processing power.

To address these limitations, the blockchain community has extensively explored sharding protocols. Sharding involves partitioning the blockchain network into smaller, independent groups called "shards," each responsible for processing a subset of transactions and maintaining its own segment of the ledger. This approach aims to distribute the workload, allowing multiple transactions to be processed in parallel across different shards, thereby increasing overall throughput and scalability with the number of participating miners.

However, existing state-of-the-art sharding protocols typically employ a uniform shard formation mechanism. This means that miners are distributed evenly across all shards, often without regard for their individual network capabilities or hardware performance. Consequently, each shard inevitably contains a mix of fast and slow miners, recreating the "struggler problem" at the shard level. This uniform distribution negates some of the potential benefits of sharding, as the throughput of each individual shard remains limited by its slowest members. Manifoldchain directly tackles this critical oversight by proposing a non-uniform, bandwidth-aware sharding strategy to unlock the true scaling potential of decentralized networks.

Key Findings

▶ Watch: Manifoldchain's block decoupling for cross-shard security (4:00)

Manifoldchain's core contribution is its innovative bandwidth-clustered sharding mechanism. Instead of uniformly distributing miners, this protocol intelligently groups miners with similar network bandwidth characteristics into the same shards. The fundamental insight is that by separating fast miners from slow "strugglers," shards composed entirely of high-bandwidth participants can operate at significantly higher transaction processing rates, thereby maximizing the network's aggregate throughput. For instance, in a simplified scenario with two fast and two slow miners, uniform sharding would place one fast and one slow miner in each shard, limiting both shards to the slow miner's pace. In contrast, Manifoldchain would create a "fast shard" and a "slow shard," allowing the fast shard to achieve a much higher mining rate, potentially one block per minute or more, compared to the uniform setup.

This non-uniform sharding, while beneficial for performance, introduces a significant security challenge: an adversary could concentrate their hashing power into a shard with fewer honest miners, compromising its security. To counteract this, Manifoldchain implements a sharing mining mechanism. Honest miners can diffuse their hashing power across shards, specifically by generating "inclusive blocks" (explained below) that contribute to the security of other shards, particularly those with a lower honest miner ratio. This ensures that even shards that might naturally have fewer honest miners due to bandwidth clustering can maintain an honest majority.

The protocol achieves this by decoupling the traditional Bitcoin block into two distinct types:

1. Consensus Blocks: These are analogous to Bitcoin's block headers, containing essential information such as the hash value, the parent block's hash, the Merkle root, and the nonce. They are lightweight and primarily serve to establish consensus and chain progression.
2. Transaction Blocks: These are akin to Bitcoin's block bodies, hosting the entire ledger and all transaction data. They are significantly larger and contain the actual payload of the blockchain.

Consensus blocks are further categorized based on their mined hash value:

• Exclusive Blocks: If a mined consensus block's hash value is greater than a predefined threshold, it is classified as an exclusive block. These blocks extend the blockchain trace within only the miner's current shard.
• Inclusive Blocks: If the hash value is smaller than the threshold, it's an inclusive block. These blocks have the unique property of being able to extend the longest trace across all shards, effectively allowing honest miners to contribute their hashing power to the security of other shards.

A critical challenge identified by the speaker is the "hashing power surrender attack." In Manifoldchain, miners act as light clients for other shards' consensus, meaning they receive inclusive blocks but do not download the full transaction blocks from those external shards. An attacker could exploit this by proposing an invalid consensus block within their own shard, which, while recognized as invalid by local honest miners, could be propagated to other shards. Miners in external shards, lacking the full transaction block for verification, might inadvertently accept this invalid block, leading to a split in honest hashing power and a compromise of consensus. To mitigate this, Manifoldchain integrates flow proof and coded democracy mechanisms, enabling miners to verify the validity and data availability of inclusive blocks from other shards without requiring the full transaction history.

Furthermore, acknowledging that verification processes can introduce latency and inhibit throughput, Manifoldchain introduces predicted mining. This technique allows miners to perform mining and verification tasks simultaneously. When unverified blocks exist in potential forks, a newly mined block extends all these unverified forks. Miners then simultaneously request coded samples and proofs for verification. Upon completion, invalid consensus blocks are rejected, and in cases where multiple valid blocks are found at the same level, a deterministic rule (e.g., choosing the block with the smaller hash value) is used to prune the alternatives, ensuring a single, verified chain.

The protocol's security and throughput claims are underpinned by theoretical analyses:

• Theorem 1: Manifoldchain is proven secure as long as the majority of the miners (specifically, over 50%) are honest. This assumption, while common in many blockchain protocols, was noted in the Q&A as a strong one in light of newer attacks.
• Theorem 2: The protocol's throughput is demonstrably scalable with the miners' bandwidth. In a worst-case scenario, Manifoldchain can achieve the same throughput as Bitcoin within each individual shard.

Experimental validation, conducted on a real-world testbed with 50,000 lines of Rust code, compared Manifoldchain against Monoxide, identified as the only other permissionless sharding protocol capable of tolerating up to 1/2 adversarial power. The results demonstrated:

• Significantly higher throughput and lower latency compared to Monoxide.
• Superior horizontal scalability, showing a greater increment in throughput as the number of miners increases.
• Achieving higher throughput with the same bandwidth resources, confirming the efficiency of bandwidth-clustered sharding.

Technical Deep Dive

▶ Watch: Solving cross-shard transaction verification with flow proof (6:00)

Manifoldchain's technical innovation begins with its unique bandwidth-clustered sharding model. Unlike traditional sharding that distributes miners uniformly, Manifoldchain dynamically groups miners based on their network bandwidth capabilities. This means that shards are not arbitrary collections of nodes but rather performance-homogeneous clusters. For example, miners with high bandwidth and low latency would form "fast shards," while those with more constrained network resources would form "slow shards." This segregation is critical because it allows fast shards to operate at their maximum potential mining rate without being held back by "strugglers," thereby dramatically increasing the overall network's transaction processing capacity. The protocol must first characterize miner bandwidth, which could involve network probing or self-reporting mechanisms (with appropriate validation to prevent Sybil attacks or misrepresentation).

Central to Manifoldchain's operation is the decoupling of the Bitcoin block structure into two fundamental components: the consensus block and the transaction block. The consensus block is a lightweight entity, analogous to a Bitcoin block header, containing only the essential metadata required for chain validation: the hash of the current block, the hash of its parent block, the Merkle root of the transactions it implicitly references, and the nonce discovered through Proof of Work. This minimal information allows for rapid propagation and consensus verification. The transaction block, in contrast, is the heavier component, containing the full set of transactions that constitute the ledger updates. This separation enables shards to quickly process and propagate consensus information without needing to download and verify large transaction payloads from other shards immediately.

Further sophistication is introduced by categorizing consensus blocks into two types based on a deterministic rule applied to their hash value:

1. Exclusive Blocks: A consensus block is deemed "exclusive" if its hash value exceeds a predefined threshold. An exclusive block extends the blockchain trace only within the shard where it was mined. This maintains the independence of shard-specific ledgers.
2. Inclusive Blocks: Conversely, if a consensus block's hash value falls below the threshold, it becomes an "inclusive" block. Inclusive blocks are pivotal for inter-shard security. They can extend the longest trace across all shards, meaning an inclusive block mined in one shard can be recognized and incorporated by other shards. This mechanism is crucial for the sharing mining strategy, where honest miners implicitly "diffuse" their hashing power across the entire network by contributing to the global chain, thereby bolstering the security of shards that might have a lower honest miner ratio due to bandwidth clustering. For instance, if a shard, due to its composition of lower-bandwidth miners, has a temporarily low honest majority, inclusive blocks originating from other, more robust shards can be accepted, raising the effective honest hashing power above the critical 50% threshold and preventing adversarial takeovers.

A significant challenge arising from this decoupled and sharded architecture is the hashing power surrender attack. Because miners in one shard primarily operate as "light clients" regarding the full transaction history of other shards, they might receive an inclusive consensus block without also downloading its corresponding full transaction block. An adversary can exploit this by proposing an invalid consensus block within their shard, knowing that local honest miners will reject it. However, the adversary can then propagate this invalid consensus block (specifically, an inclusive one) to other shards. Miners in these external shards, lacking the full transaction block to verify its contents, might mistakenly accept it, leading to a split in the honest miners' hashing power across different, potentially invalid, chain traces. This fragmentation compromises the network's security and consensus integrity.

To counter this, Manifoldchain employs flow proof and coded democracy. While the speaker did not delve into the intricate details due to time constraints, these mechanisms are designed to allow miners to verify two crucial properties of an incoming inclusive block from another shard: transaction validity and data availability. This verification occurs without requiring the miner to download the entire transaction block or the full historical ledger of the originating shard. Flow proof likely involves cryptographic proofs that attest to the validity of the transactions summarized in the Merkle root of the inclusive block. Coded democracy, possibly leveraging erasure coding or similar techniques, ensures that the transaction data is available across the network even if not directly downloaded by every verifying node, preventing an attacker from withholding data. These techniques are essential for maintaining inter-shard trust and preventing the "hashing power surrender" attack.

Finally, to address the inherent latency introduced by verification processes, Manifoldchain integrates predicted mining. In traditional blockchain systems, mining typically pauses or slows down while waiting for previous blocks to be fully verified. This introduces idle time and reduces throughput. Predicted mining allows miners to conduct mining and verification simultaneously. In a scenario where multiple unverified blocks exist, forming potential forks, a miner can optimistically extend all these unverified forks with a newly mined block. As these new blocks are propagated, miners simultaneously request the necessary "coded samples and proofs" (related to flow proof and coded democracy) to verify the underlying blocks. Once verification is complete, any consensus blocks found to be invalid are simply rejected, and their respective forks are pruned. If multiple valid consensus blocks are verified at the same level (a rare but possible occurrence in a fork scenario), a deterministic rule—such as choosing the block with the smallest hash value—is applied to select the canonical block, ensuring that eventually, only one valid consensus block is accepted at any given level. This concurrent approach significantly reduces the impact of verification latency on overall system throughput.

Demo / Proof of Concept

▶ Watch: Predicted mining for simultaneous mining and verification (8:00)

While the talk did not feature a live demonstration in the traditional sense, the speaker confirmed that the Manifoldchain protocol has been thoroughly implemented and evaluated. The team developed a complete protocol implementation comprising 50,000 lines of Rust code. This extensive codebase was then deployed and tested on a real-world testbed, providing empirical validation for the theoretical claims. The implementation and subsequent experimental evaluation serve as a robust proof of concept, demonstrating the practical feasibility and performance advantages of Manifoldchain's design. The results from these experiments were instrumental in comparing Manifoldchain against existing sharding solutions and confirming its enhanced throughput and scalability.

Defensive Implications

▶ Watch: Security analysis, throughput scalability, and experiments (9:00)

Manifoldchain presents several critical defensive implications for blockchain developers, architects, and security professionals seeking to build more scalable and resilient decentralized systems.

Firstly, the core concept of bandwidth-clustered sharding offers a powerful paradigm shift. Defenders should move beyond simplistic uniform distribution models and explore how network characteristics and miner performance can be leveraged to optimize throughput. This requires robust mechanisms for profiling miner capabilities and dynamically assigning them to shards, potentially requiring novel consensus algorithms for shard formation and maintenance. Integrating such a performance-aware sharding strategy could unlock significant scalability gains for future blockchain applications.

Secondly, the protocol highlights the complex security challenges introduced by inter-shard communication and decoupled block structures. The hashing power surrender attack is a clear warning that simply passing consensus blocks between shards without full transaction verification can lead to severe vulnerabilities. Defenders must implement sophisticated cross-shard verification mechanisms, such as Manifoldchain's proposed flow proof and coded democracy, to ensure the validity and data availability of transactions originating from other shards. This requires designing efficient cryptographic proofs and data availability schemes that do not necessitate downloading the entire state of every shard, which would negate the benefits of sharding.

Thirdly, the concept of predicted mining offers a blueprint for mitigating latency introduced by complex verification processes. As blockchain protocols become more sophisticated, the time required to validate blocks can become a bottleneck. Defenders should investigate and implement optimistic or concurrent verification strategies that allow mining and verification to occur in parallel, reducing idle time and maximizing throughput. This could involve speculative execution, pipelining, or other techniques to overlap computational tasks.

Finally, the discussion during the Q&A session regarding the 50%+ honest majority assumption is a crucial takeaway for security architects. While Manifoldchain, like Bitcoin, relies on this fundamental assumption, the questioner correctly pointed out that newer research indicates potential attacks with significantly less adversarial hashing power (e.g., 26%). This underscores that even highly optimized sharding protocols do not obviate the need for robust security foundations. Defenders must critically evaluate the security assumptions of any sharding design, consider the implications of minority attacks, and potentially integrate additional layers of security, such as economic incentives, reputation systems, or more advanced consensus mechanisms, to enhance resilience against sophisticated adversaries. Manifoldchain provides a framework for scaling, but its security, like many decentralized systems, ultimately rests on the collective honesty of its participants and the continuous evolution of attack mitigation strategies.

Key Takeaways

• Blockchain throughput is severely limited by highly overlapping miner tasks and the "struggler problem," where slow miners bottleneck the entire network.
• Manifoldchain introduces bandwidth-clustered sharding, an innovative approach that groups miners with similar network performance into dedicated shards, allowing high-performance shards to achieve significantly higher mining rates.
• To secure non-uniform shards, Manifoldchain employs sharing mining via inclusive blocks, enabling honest miners to diffuse their hashing power across the network and bolster the security of potentially weaker shards.
• The protocol decouples blocks into lightweight consensus blocks (for fast propagation and consensus) and heavy transaction blocks (for ledger data), further categorizing consensus blocks into exclusive (shard-local) and inclusive (cross-shard) types.
• Advanced techniques like flow proof and coded democracy are essential to verify transaction validity and data availability of cross-shard blocks without downloading full transaction histories, mitigating attacks like the "hashing power surrender."
• Predicted mining optimizes throughput by allowing mining and verification to occur simultaneously, reducing latency and maximizing the utilization of network resources.
• Experimental results demonstrate that Manifoldchain, implemented in 50,000 lines of Rust code, achieves superior throughput and horizontal scalability compared to existing sharding protocols like Monoxide, validated on a real-world testbed.

About the Speaker(s)

Chunjiang Che is a researcher who presented the work on Manifoldchain at the NDSS Symposium. Based on the technical depth and reference to "our paper," the work likely originated from an academic or research institution focused on blockchain scalability and security. His presentation highlighted deep expertise in distributed systems, cryptography, and blockchain architecture.