The ByzRP Solution: A Global Operational Shield for RPKI Validators

The ByzRP Solution: A Global Operational Shield for RPKI Validators

Speakers: [Information not provided in bundle]

Conference: Black Hat Asia

YouTube: https://www.youtube.com/watch?v=3eZ0VEMgEUg

Overview

This talk introduces ByzRP (Byzantine Fault Tolerant RPKI), an innovative approach to enhance the security, robustness, and performance of the Resource Public Key Infrastructure (RPKI). RPKI is a critical component for securing the internet's routing infrastructure, specifically designed to mitigate Border Gateway Protocol (BGP) hijacking attacks. While RPKI has seen significant adoption by major Internet Service Providers (ISPs) and companies like Amazon, and has even been highlighted in U.S. government roadmaps for internet security, its current implementation faces several inherent vulnerabilities and operational challenges.

The speakers highlight that existing RPKI relying parties (RPs) – the software engines that collect, validate, and disseminate RPKI data – are susceptible to denial-of-service (DOS) and stalling attacks, and their outputs often suffer from instability. ByzRP addresses these issues by integrating a watchdog mechanism into RPs and deploying them within a Byzantine Fault Tolerant (BFT) consensus network. This distributed architecture aims to provide a globally robust and consistent source of validated RPKI data, drastically reduce network traffic, and simplify RPKI adoption across the internet.

The significance of ByzRP lies in its potential to strengthen the foundational security of internet routing at a time when BGP hijacking remains a persistent and costly threat. By offering a more resilient and efficient RPKI validation system, ByzRP not only protects network operators from malicious actors but also streamlines the deployment of RPKI, potentially leading to broader adoption and a more secure global internet.

Background

▶ Watch: Understanding how BGP hijacking attacks work (0:48)

The internet's core routing mechanism, the Border Gateway Protocol (BGP), was designed in the early 1990s with priorities on scalability, efficiency, and speed, but notably lacking in inherent security. This fundamental design flaw makes BGP vulnerable to hijacking attacks, where a malicious actor advertises ownership of a network prefix they do not control. Routers, making decisions based on "neutral heuristics" like shortest path, can then inadvertently divert traffic to the attacker, leading to service disruption, data interception, or other malicious activities. Such attacks are common and carry a hefty price tag for victims.

To counter these vulnerabilities, the Resource Public Key Infrastructure (RPKI) was introduced and has emerged as the most widely deployed solution. RPKI works by enabling rightful owners of network prefixes to create cryptographically signed bindings between their network prefix and their Autonomous System Number (ASN). These bindings, known as Route Origin Authorizations (ROAs), are stored in distributed publication points (PPs) across the internet. Network operators wishing to protect their infrastructure install relying party (RP) software. The RP collects ROAs from various PPs, performs cryptographic validation to verify their authenticity, and then generates a list of Validated ROA Payloads (VRPs) – tuples of prefix and ASN bindings that are confirmed to be valid. This list serves as a "ground truth" for routers, allowing them to evaluate BGP announcements and identify potential hijacking attempts. RPKI has gained significant traction, with major Tier-1 ISPs and large companies like Amazon enforcing its use, and even the FCC and the US White House highlighting it as a crucial component for national internet security.

Despite its growing importance, the current RPKI infrastructure suffers from several significant shortcomings. The talk identifies three primary issues:

1. Malicious Publication Points and Stalling Attacks: A compromised PP can orchestrate a stalling attack by slowing down the RP's data collection process. This can cause legitimate data in the RP's cache to expire, leading to an incomplete or outdated view of the RPKI ground truth for routers. The speakers referenced a Black Hat 2022 talk by their team for more details on this attack vector.
2. Denial-of-Service (DOS) Attacks on Relying Parties: Malicious entities can become PPs (a process with a low entry barrier) and publish malformed files designed to crash RPs. When an RP crashes, it ceases to produce output, effectively downgrading routers to an RPKI-unprotected state. Measurements cited from a previous year's talk revealed that at least 50% of all RPs on the internet were vulnerable to DOS attacks due to processing issues or outdated implementations.
3. Network and Repository Instability: Even without overt attacks, the RPKI ecosystem exhibits significant instability. The speakers observed "oscillations" in RP output, with VRP counts fluctuating (e.g., "+3, -50, +300, -500"). This instability stems from factors like network instability, repository outages, connection drops (some PPs being overwhelmed), and frequent content updates (addition/deletion of data). The consequence is that different RPs, or even the same RP at different times, produce slightly disparate outputs, leading to a lack of uniformity in the ground truth received by routers – an undesirable state for a security system.

These shortcomings underscore the need for a more robust and resilient RPKI infrastructure, which ByzRP aims to provide.

Key Findings

▶ Watch: RPKI's widespread adoption and future importance (4:40)

The research and development behind ByzRP yielded several critical findings regarding the state of RPKI and the potential for a more secure, robust, and efficient alternative:

• Widespread Vulnerability of Current RPKI RPs: A significant portion of currently deployed relying parties (RPs) are vulnerable to both stalling and denial-of-service (DOS) attacks. Measurements indicated that at least 50% of RPs were susceptible to DOS attacks due to outdated implementations or processing flaws. This highlights a critical weakness in the existing RPKI security chain.
• Inherent Instability in RPKI Data Distribution: Even in benign environments, the output of individual RPs is inherently unstable and oscillates. This instability is attributed to transient network issues, publication point outages, connection drops, and frequent content changes. This leads to disparate RPKI ground truth being supplied to different routers, undermining the consistency required for robust routing security.
• Effectiveness of the Watchdog Mechanism: Integrating a watchdog into RPs can effectively mitigate stalling and DOS attacks. Simulations demonstrated that the watchdog can detect anomalies, identify malicious publication points (PPs), add them to a skip list, and reboot the RP, resulting in virtually no operational downtime. While the number of VRPs might temporarily drop due to skipped malicious PPs, the system remains functional and continues to process valid data.
• Superior Robustness through Consensus Aggregation: Deploying multiple enhanced RPs (with watchdogs) in parallel and aggregating their outputs via majority voting significantly enhances the overall system's robustness. This approach makes the ByzRP network impervious to transient network failures that would affect individual nodes, maintaining a stable and comprehensive output.
• Dramatic Reduction in Network Traffic: A centralized ByzRP service model, replacing thousands of individual RPs with a smaller, distributed network (e.g., 15 nodes), can achieve a 98-99% reduction in RPKI-related network traffic. This is because the bulk of traffic comes from individual RPs downloading large cryptographic objects from PPs. This reduction also enables 10 times more frequent updates of RPKI data, ensuring routers operate with the most current information.
• Byzantine Fault Tolerance for Distributed Trust: The use of a Byzantine Fault Tolerant (BFT) consensus algorithm allows ByzRP to operate securely even if a minority of its nodes are compromised or malicious. This distributes trust across the network, eliminating the need to trust any single entity and making the system resilient to targeted attacks on individual nodes.
• Simplified RPKI Adoption: ByzRP can be offered as a service, simplifying RPKI adoption for network operators who would no longer need to set up, maintain, and patch their own RPs. They would simply point their routers to the ByzRP service, lowering the barrier to entry for RPKI deployment.

In essence, the key findings demonstrate that ByzRP transforms RPKI from a vulnerable, unstable, and traffic-heavy system into a highly robust, performant, and uniformly secure service, making it more accessible and reliable for global internet routing security.

Technical Deep Dive

▶ Watch: Introducing ByzRP: A new RPKI infrastructure (5:50)

The ByzRP solution fundamentally re-architects the RPKI validation process by combining enhanced relying party (RP) functionality with a Byzantine Fault Tolerant (BFT) consensus network.

The first layer of enhancement involves bolstering individual RPs with a watchdog mechanism. This watchdog operates in parallel with the RP, continuously monitoring its well-being, health, and its connections to publication points (PPs). Its primary functions are:

• Health Monitoring: Checking if the RP is operating correctly and detecting crashes.
• Connection Monitoring: Observing the duration and outcome of connections established with PPs.
• Adaptive Skip Listing: If the watchdog detects an anomaly (e.g., a connection lasting too long indicating a stalling attack, or a connection causing the RP to crash), it identifies the offending PP. That PP is then added to a skip list, and the RP is immediately rebooted. Upon reboot, the RP will bypass any PPs on its skip list, preventing further compromise or disruption from that source. This mechanism ensures rapid recovery and sustained operation, minimizing downtime to mere seconds.

To address the observed instability and disparity in RP outputs, ByzRP deploys these enhanced RPs (each with its watchdog) as independent nodes within a distributed network. These nodes can run on different containers, hosts, or virtual machines, completely isolated from one another. Each node independently collects and validates RPKI data, producing its own set of Validated ROA Payloads (VRPs).

The core innovation lies in the consensus aggregation layer, which ensures uniformity and maximizes the number of verifiable VRPs. The system employs a Byzantine Fault Tolerant (BFT) algorithm, a concept borrowed from distributed computing, particularly relevant in environments where not all participants can be fully trusted (e.g., blockchain networks). BFT ensures that a network can reach a correct consensus even if some nodes are malicious or compromised.

In ByzRP's implementation:

1. Network Topology: Nodes form a fully connected network, meaning each node communicates directly with every other node. This avoids reliance on broadcast mechanisms and ensures message delivery.
2. Secure Communication: Communication between nodes is secured using mutual TLS, authenticating both clients and servers.
3. Asynchronous Polling: Each node runs a small web server to serve its current VRP output. Other nodes asynchronously poll their peers to retrieve their respective outputs at any given time.
4. Independent Consensus Calculation: Critically, each node independently calculates the network's overall output. Since VRPs are independent objects, their outputs can be treated as sets. The consensus is reached by intersecting these sets based on a predefined threshold.

The threshold for consensus is a configurable parameter, offering a trade-off between robustness (resistance to benign failures and censorship) and resistance to poisoning (malicious inclusion of invalid data):

• Union (Threshold = 0): If the threshold is set to zero, an object is included in the final output if it is seen by at least one node. This is maximally robust and censorship-resistant but means trusting every node and is highly vulnerable to poisoning (a single malicious node could inject invalid VRPs).
• Intersection (Threshold = N-1): If the threshold is set to N-1 (where N is the total number of nodes), an object is only included if it is seen by all N nodes. This is maximally resistant to poisoning (requiring all nodes to be compromised to inject invalid data) but loses the robustness benefits, as a single benign failure or outage would prevent consensus for that object.
• Majority Voting (e.g., Threshold = N/2 + 1): ByzRP typically opts for a majority vote (e.g., 2 out of 3, or 3 out of 5 nodes). This balances the need for faster inclusion of new, legitimate objects with faster removal of outdated or invalid ones, while maintaining resilience against a minority of malicious or failing nodes. For example, in a 5-node network, a 3-out-of-5 vote ensures that the system is resilient to up to two malicious nodes. If two nodes attempt to inject a malicious VRP, it won't be included if the other three benign nodes don't see it. Conversely, if two nodes fail, the remaining three can still reach consensus.

This same BFT consensus mechanism is also applied to the skip list entries. If a watchdog on one node identifies a potentially malicious PP, this information can be shared across the network. If a majority of nodes encounter similar problems with the same PP, a network-wide consensus can be reached to globally skip that PP, improving overall performance and accuracy by avoiding localized "flukes."

In summary, ByzRP transforms the RPKI validation process into a distributed, self-healing, and resilient service. It protects against internal and external threats, ensures consistency across the internet, and leverages cryptographic and distributed systems principles to build a truly robust operational shield.

Demo / Proof of Concept

▶ Watch: Vulnerability 1: Stalling attacks from malicious publication points (6:20)

The speakers presented several simulations and measurements to demonstrate ByzRP's capabilities and validate its design principles.

1. DOS Attack Simulation and Watchdog Effectiveness:

• A simulation of a DOS attack on an RPKI RP was performed. The graph showed the number of VRPs (Validated ROA Payloads) output by the system over time.
• Initially, the system warms up, reaching a peak output of approximately 517,000 VRPs.
• After 15 minutes, a crash was simulated by introducing a malicious publication point (PP).
• In a standard RP, this would cause the output line to drop to zero and remain there until manual intervention.
• With ByzRP's watchdog mechanism, a drop in VRPs was observed, but the system quickly stabilized at a slightly lower level. The purple line, representing ByzRP's output, remained largely stable, indicating no operational downtime. The drop represented the VRPs that would have been processed by the now-skipped malicious PP, but all other valid data continued to be served. This demonstrated the watchdog's ability to detect, isolate, and recover from such attacks automatically.

1. Robustness Against Transient Failures:

• The talk presented graphs comparing the stability of individual ByzRP nodes (blue lines) versus the aggregated output of the ByzRP system as a whole (purple line).
• Individual nodes frequently showed momentary outages or dips in their VRP output due to transient network failures or repository issues.
• However, the aggregated ByzRP output (purple line) remained remarkably stable, with no significant peaks or troughs below its baseline. This illustrated how the consensus aggregation mechanism effectively smooths out sporadic, non-systemic failures experienced by individual nodes, making the entire system impervious to such transient issues. This was shown for 24-hour measurement periods.

1. Fast Convergence:

• Despite being a distributed system, ByzRP demonstrated fast convergence. After the initial booting, all nodes in the network reached consensus and stabilized their output within less than 10 minutes, typically 5 minutes or less, with a worst-case scenario of up to 6 minutes. Once converged, the system remained stable without large-scale outages or resynchronization events.

1. Dramatic Traffic Reduction:

• The most compelling quantitative proof concerned network traffic. The speakers provided current and extrapolated future figures for RPKI traffic:
• Current RPKI Ecosystem (as of study last year):
• 64 publication points (PPs).
• Total cryptographic objects: ~500 MB (compressed data).
• ~3,000 relying parties (RPs) contacting a PP in a 24-hour period, each downloading 500 MB.
• Validated output size: ~6 MB per RP.
• Assuming one RP per router, this results in nearly 2 terabytes (TB) of data traffic for each validation round.
• ByzRP Network (15-node example): Replacing the 3,000 individual RPs with a 15-node ByzRP network would reduce this traffic by approximately 98%. The bulk of the traffic (downloading 500MB objects) would only occur 15 times, not 3,000.
• Future Full RPKI Deployment (extrapolated):
• Assumed 128,000 RPs (one per router for every network).
• Assumed ROA data growth to 1.2 GB.
• This would lead to over 150 TB of traffic per validation round.
• A single ByzRP network would achieve a 99% reduction in this scenario, allowing for 10 times more frequent updates while still using only a fraction of the traffic.

These demonstrations collectively prove ByzRP's ability to deliver a highly robust, performant, and efficient RPKI validation service, significantly overcoming the limitations of current deployments.

Defensive Implications

▶ Watch: Vulnerability 2: DOS attacks crashing RPKI relaying parties (7:00)

ByzRP offers several crucial defensive implications for network operators and the broader internet ecosystem, addressing both operational resilience and strategic security posture.

1. Enhanced Resilience Against RPKI Attacks:

• Stalling and DOS Attack Mitigation: The integrated watchdog mechanism provides automated, real-time protection against malicious publication points (PPs) attempting to stall or crash relying parties (RPs). This means network operators utilizing ByzRP are shielded from RPKI infrastructure-level attacks that currently affect a significant portion of RPs (estimated at 50% vulnerability).
• Self-Healing Capabilities: The ability of ByzRP nodes to identify offending PPs, add them to a skip list, and immediately reboot ensures continuous operation with minimal downtime (seconds), preventing prolonged RPKI downgrades that would expose routers to hijacking.

1. Consistent and Comprehensive RPKI Ground Truth:

• Uniformity of Data: ByzRP's Byzantine Fault Tolerant (BFT) consensus aggregation ensures that all routers connected to the ByzRP service receive the exact same, validated RPKI data. This eliminates the problem of disparate ground truths caused by network instability, providing a consistent and trustworthy basis for Route Origin Validation (ROV) decisions.
• Maximizing Verifiable VRPs: The consensus mechanism, particularly with majority voting, effectively filters out transient failures from individual nodes, ensuring that the aggregated output maximizes the number of legitimate Validated ROA Payloads (VRPs), providing a more complete picture of the valid routing space.

1. Distributed Trust and Attack Surface Reduction:

• Resilience to Node Compromise: The BFT design means that network operators do not need to trust any single ByzRP node or the entity operating it. The system remains secure and provides correct output as long as a majority of nodes are honest. This significantly raises the bar for attackers; compromising a single node or even a minority of nodes will not subvert the entire RPKI validation process.
• Protection Against Centralization Concerns: While the concept might appear to centralize RPKI validation, the BFT nature ensures distributed trust. The speakers clarified in Q&A that while a 15-node network was an example, the service could be offered by multiple distinct entities, further distributing the trust and attack surface. DDoS attacks against the ByzRP network itself would need to overcome standard DDoS protection measures, similar to other critical internet services like DNS root servers.

1. Operational Efficiency and Simplified Adoption:

• Reduced Operational Overhead: For network operators, ByzRP can be offered as a service. This eliminates the need to deploy, configure, maintain, and regularly patch their own RPs, which is a significant barrier to RPKI adoption. Operators would simply point their routers to the ByzRP service.
• Significant Traffic Reduction: The drastic reduction in RPKI-related network traffic (98-99%) frees up bandwidth and reduces the load on publication points, making the entire RPKI ecosystem more efficient and scalable.
• More Up-to-Date Routing Decisions: The reduced traffic overhead allows for 10 times more frequent updates of RPKI data. This means routers can make routing decisions based on the most current information, further enhancing security and responsiveness to changes in the routing landscape.

In essence, ByzRP provides a robust, efficient, and trustworthy mechanism for RPKI validation, empowering defenders with better tools to combat BGP hijacking and simplifying the path to widespread RPKI adoption across the internet.

Key Takeaways

• Current RPKI Deployments Are Vulnerable and Unstable: Existing RPKI relying parties (RPs) are highly susceptible to denial-of-service (DOS) and stalling attacks (50% vulnerability observed), and their outputs suffer from instability and inconsistency due to network and repository issues.
• ByzRP Enhances Individual RPs with a Watchdog: The ByzRP solution integrates a watchdog mechanism into RPs to detect and mitigate attacks from malicious publication points (PPs), enabling rapid recovery and continuous operation with minimal downtime.
• Byzantine Fault Tolerance (BFT) Ensures Robustness and Uniformity: ByzRP deploys enhanced RPs in a distributed network and leverages a BFT consensus algorithm to aggregate their outputs, providing a globally uniform, comprehensive, and stable RPKI ground truth for all connected routers.
• Dramatic Traffic Reduction and Increased Update Frequency: A ByzRP service model can reduce RPKI-related network traffic by 98-99%, allowing for significantly more frequent updates (e.g., 10 times more often) and ensuring routers operate with the most current routing security information.
• Distributed Trust Model Simplifies Adoption: ByzRP's BFT design distributes trust across multiple entities, eliminating the need to trust any single node. This enables RPKI to be offered "as a service," simplifying its adoption for network operators who no longer need to manage their own RPs.
• Overall: More Robustness, Better Performance, Higher RPKI Adoption: ByzRP is designed to make the RPKI infrastructure more resilient to attacks, more performant in terms of traffic and update frequency, and ultimately easier to deploy, thereby fostering wider RPKI adoption and a more secure internet.

About the Speaker(s)

The provided talk bundle does not include specific speaker names, titles, or affiliations. The presentation refers to "our team" and mentions previous Black Hat talks in 2022 and last year (2023), indicating a continuous research effort by a group or institution in the field of RPKI security. The speakers demonstrate deep technical expertise in BGP, RPKI infrastructure, distributed systems, and Byzantine Fault Tolerance. They also reference a paper published at CCS last year on the topic, suggesting an academic or research background for the team behind ByzRP.